Search Unity

It’s time. It’s been a long run, but here we are: in Unity 2017.1 the Playable API is out of experimental.

The graph-like Playable API allows you to have precise programmatic control over animation and audio – we are also working on using the same API for video control. Our brand new Timeline tool in 2017.1 uses the Playable API.

For animation, this provides a level of control that was not possible before with the Animator:

  • Play an AnimationClip on an Animator without an AnimatorController
  • Direct control of the timing and the weight of each individual AnimationClip being played
  • Dynamically add AnimationClips to the graph at runtime
  • Mix AnimatorController with AnimationClips, other AnimatorControllers, or even with Timeline!

I wrote a blog post sometime ago regarding this, which you can read here.

In order to make it easier for users to create and debug, we created a PlayableGraph Visualizer tool. The tool can be used to display any PlayableGraph in both Play and Edit mode and will always reflect the current state of the graph. Playables in the graph are represented by colored nodes, varying according to their type. Wire color intensity indicates the local weight of the blending. You can get the visualizer tool on our github.

Note that the API has changed a bit from 5.6 to 2017.1. We’ve just finished writing a manual so you can see what it’s all about.

Something that is really interesting about this API is that for all “native” playables that Unity provides, we use C# structs instead of C++ objects to hold the objects. Using structs has the advantage of not allocating GC memory. It can be a bit more tricky to use, but since we are building a lot of our future technologies with this API this was important for us.

It’s now possible for our users to implement and tailor their own systems and API that rely on Playables. For instance, for those who are still fond of the Legacy Animation system, we are currently implementing – using Playable API – a frontend API that mimics the behavior of the Legacy system but using Mecanim. This will soon be available via our GitHub. For now, let’s just see a small example:

With this, you get the benefits of our multithreaded and retargetable animation engine but with a Legacy-like level of control.

PlayableBehaviour

When the Playables API was still in the experimental phase we had something called ScriptPlayable. It was both the container (the playable node) and the content (your custom code). We’ve since split this concept in two: the ScriptPlayable is now just the container and the PlayableBehaviour is the content.

From the manual we have :

The Playables API allows you to create custom playables derived from PlayableBehaviour. By overriding the ‘PrepareFrame’ method, nodes can be handled as desired. Custom Playables can also override any of the other virtual methods of PlayableBehaviour, based on the event that they need to handle.

For instance, a basic Playable would look like this:


And for its creation, we do this:


This is a bit different from what we did in the previous Playable version, but we believe that this aligns better with what we have planned for the future. And speaking of the future…

The future

Joachim recently gave a presentation about of C# compute jobs. Go watch this presentation it’s really good stuff.

We are working, as we speak, to bring AnimationPlayables to the C# jobs system. This will allow users to directly control the animation stream values (transforms, floats, muscle, etc.) in an optimized, multithreaded and thread-safe C# code. The user-made C# animation jobs are then connectable in the PlayableGraph and allows you to create custom graph based rigs/constraints.

The code, written in C#, will run in our multithreaded animation system and will be evaluated during the traversal of the Playable graph.

As a short example, this is how we do a very simple FullBodyIK algorithm.

 


The setup/instantiation of the playable looks like this :


 

Note that this code is still an early prototype and is subject to change.

We also have prototyped the following use-cases:

  • CustomMixer: allows to have different weights per transform during mixing
  • IKSolver: custom made IK solver; currently 2-bone, can be CCD (https://sites.google.com/site/auraliusproject/ccd-algorithm)  or other.
  • MotionStream: a special node that plays poses directly (instead of animation)
  • LiveMocap: reads mocap and writes it into the AnimationStream
  • KneePoppingFixer: adds scale when we are near full extension of body part

Theses are nice examples of what we can do with the C# AnimationPlayables, and we can’t wait to let you get your hands on this. Stay tuned!

48 Comments

Subscribe to comments

Comments are closed.

  1. Hey guys, what’s the ETA on those legacy API to playable API wrappers?

  2. For instance, for those who are still fond of the Legacy Animation system, we are currently implementing – using Playable API – a frontend API that mimics the behavior of the Legacy system but using Mecanim. This will soon be available via our GitHub.

    Guys, we are still waiting… /:

  3. Hi PP,
    thanks for the great blogs!
    Can we please get an example of how to create custom anim and anim controller tracks for Timeline, it would be extremely useful. I tried really hard but couldn’t make it work, it’s really not obvious where plug code in there, too much under-the-hood code is hidden from us.

    Thanks!

    Manu

    1. Hi Manuel

      Here are some resources you can look at:

      Extending Timeline with your own playables: A talk by Unity’s James Bouckley about making custom playables.
      Default playables on the Asset Store. These examples could help you getting on the right track (pun intended).

      You can also head to the Unity Timeline forums if you have specific questions; the timeline developers will be there to help you.

      Jul

      1. Hi Julien,
        thanks, I watched the presentation and downloaded the default playables a couple of weeks ago, but none of them go far enough for what we want to do, they all cover very simple custom tracks. I think only someone like PP has the knowledge we need.

        Thanks!

        Manuel

      2. I’ll try the forum, thanks!

  4. I still can’t play a playable directly on the Animator (no Play() overload yet) , even through it’s been over a year since your first sample code.

    What’s worse is that now I have to pass in a graph whenever I create a playable, why not just create a Graph editor component for the heck of it now?

    Mecanim acquisition is the worst thing that happened to Unity, shoving a lame FSM over a general-purpose game engine’s entire animation system. Can’t even have a Ping-pong wrapmode on a clip or proper loop, transitions suck, root motion sucks, the workflow SUCKS, clip references are a mess.

    You won’t make it better with playables on top of it, even a gigantic graph can’t still procedurally tween a simple cube.

    When we’ll finally have a simple handy tween engine? After you ship your AlienoidMocapControllerMixerTreeRunnable in 2050?!

    Is it too much asking for something like this for God’s sake:
    Tween.LookAt (target, 2f, 3f, Ease.Linear).onComplete(()=>print (“done?”) ;

    1. Pierre-Paul Giroux

      August 28, 2017 at 11:17 pm

      > I still can’t play a playable directly on the Animator (no Play() overload yet) , even through it’s been over a year since your first sample code.

      We have AnimationPlayableUtilities.Play to do this. Since the Graph is general purpose and can have multiple outputs, we decided to use this approach instead.

      > What’s worse is that now I have to pass in a graph whenever I create a playable, why not just create a Graph editor component for the heck of it now?
      We are working on this. This API is the base building block of it.

      For the other comments…. well … hum…. lets see in 2050 where we are at! …

  5. Hi,

    Looks cool,
    I have a few questions about Playable API.
    1. In the Playable API I can control the time of AnimationClipPlayable, Is any possibility to control the time of AnimationMixerPlayable?
    2. Will there be an opportunity to get from the graph the current playable clip?

    1. Pierre-Paul Giroux

      August 16, 2017 at 1:11 pm

      hi!

      1/ the mixers are not depedent on time. Why would you like to set the time on them ? Is it because you want time to trickle down to its connected input ?

      2/ use AnimationClipPlayable.clip ! We just found out that there was a bug our online documentation, we are fixing that asap.

  6. Never liked mechanim and never used it. Still waiting for the Legacy-Like playables.

    1. Pierre-Paul Giroux

      August 16, 2017 at 1:09 pm

      hi.
      what part ? the StateMachine ? the retargeting engine ? the importers ( with auto-looping etc) ? the root motion implementatio ?

      Im honestly curious to get more details on this.

  7. This is amazing stuff. I think I’m finally starting to get it. I have a couple of things I’m not clear on yet, though.

    1) So is the timeline a separate concept from playables in general? As far as I can tell the timeline is just a playable asset that you can use to play back a bunch of other playables. Each track is a playable that can play back a number of animation clips (also playables). It gives you a visual way to arrange a sequence of playable events and clips in the editor, however, if you’re creating playable instances at runtime within scripts, the timeline has nothing to do with it, unless you’re explicitly creating timeline assets? Is that all accurate?

    2) When using clipcaps blending, where are the blending curves in the inspector coming from? As in, where is the code that exposes those values, and is this a purely timeline related concept (assuming I’m correct in question 1)?

    3) If you’re not using the timeline, a playable has no concept of clip-in or duration, correct?

    4) If I’m correct about question 1, are there any plans to expand the graph visualizer into something that can be used like the timeline to assemble not linear playable graphs and connect inputs?

    Thanks for any help. This is really exciting stuff.

    1. One additional question.
      5) PlayableExtensions SetTime and SetPlayState – it’s up to me if I create a custom playable to implement a local time property, and make my playable behave properly according to playstate, correct? There’s no actual localTime and playState variables unless I create them? Or is there a certain class I should be inheriting from in most cases that already has these?

      1. Romain Failliot

        August 8, 2017 at 1:31 pm

        Hi!

        1) That’s all accurate! ;)

        2) All these informations are Timeline related concepts that are translated into a PlayableGraph later on. For instance, the blend-in/out curves of the animation clips are translated into weights in the AnimationMixerPlayable.

        3) A Playable has no concept of clip-in (yet), but it already has a concept of duration (see Set/GetDuration). The trick is that the duration of the Playable is unrelated to the duration of the clip in it. For instance you can have an AnimationClipPlayable with duration of 10 seconds, but with an animation of 2 seconds (meaning that the animation will loop 5 times).

        4) Right now the GraphVisualizer is only a visualizer, there are no plan to be able to edit graph with it. Though I like the idea, for now I think the most important is to bring the users a way to visualize/debug their graphs, because graphs created by code can be far more tricky than the ones created by the Timeline ;)

        5) Set/GetTime and Set/GetPlayState are already local. You’ll be able to have the root time in PrepareFrame, thanks to the rootTime property of the FrameInfo.

        1. Romain Failliot

          August 8, 2017 at 2:58 pm

          Hah, didn’t see Pierre-Paul’s answer, sorry.

        2. Thanks for the answers! Sorry I didn’t see there was a reply until now.

    2. Pierre-Paul Giroux

      August 7, 2017 at 1:07 pm

      Hi there!

      1/ TimelineAsset ( ie a PlayableAsset ) generates a Playable, the Timeline. This Playable, in PrepareFrame, does its internal logic that creates child Playables, and mixes them accordingly to the asset. If you create Playable from code, you can always mix it with the Timeline by connection the proper mixe ( AnimationMixerPlayable or ) to one of it’s output.

      2/ what do you mean by ClipCaps ?

      3/ ClipIn is Timeline only at the moment. We might bring it to Playable directly… discussing this with the team. Duration is part of Playable.

      4/ We are putting a lot of effort to bring visual programming to you guys. Part of this is to bring the ability to author PlayableAsset with a graph interface.

      5/ There is Time and PlayState in the Playable. Then its up to you in PrepareFrame to do proper logic based on them.

      1. Pierre-Paul Giroux

        August 7, 2017 at 8:38 pm

        2/ got distracted here. ClipClaps is only used by Timeline.

        1. I had been checking back for replies daily but I suppose my browser had the page cached so I only saw this now. Thanks for the answers.

  8. Just please make that node graph zoomable :)

  9. Can’t wait for the legacy-like animation component! \o/

  10. This is really great!

    I have a few questions:

    (a) why the “Playable” wrapper? Since v2017 is starting to change things, why not creating a base playable class and make old ones like Animation and Audio clips inherit from it from v2017 and on? (so that as to handle less “types”), and

    (b) once you create a playable (for instance, an AnimationClipPlayable), and pass it to the graph, is it possible to assign a new AnimationClip to the AnimationClipPlayable? Or do we need to create a new instance of an AnimationClipPlayable and pass it to the graph?

    Thanks in advance for your response,
    Pete

    1. Pierre-Paul Giroux

      August 3, 2017 at 7:52 pm

      Hi!

      (a). Not sure i get the question here. Since we used c# structs without inheritance ( to prevent boxing, aka GC alloc) we used templated C# extensions methods for implementing “pseudo-inheritance”

      (b) you can set the animationClip afterwards.

      1. (b) Good to know!

        Regarding (a), I wasn’t mean for boxing issues (but even if there is no boxing, structs can be located on the heap depending on where they are declared). My concern is related to the fact that a new structure is added (the wrapping struct), instead of directly pass the existing “playable” ones to the graph with a common “inner” base operations/interface, that is, to spare the “go between”.

        1. Structs can be located on the heap, but the point is, they don’t have to be – it is possible to work with Playables without allocating any GC memory at all. That wouldn’t be possible with classes. However, structs in C# don’t support inheritance.

        2. I know that structs don’t support inheritance in C#. My point is, since you are starting to redesign key parts of the core engine, maybe this could have been solved with no wrapper at all, no matter whether it was a class or, as in this case, a struct, by redesigning existing classes like AnimationClip and AudioClip so that they inherit the “playable” stuff from a common base class from v2017 and on. This way would have got rid of any wrapper and directly pass the AnimationClip instance to the graph.

        3. Pierre-Paul Giroux

          August 7, 2017 at 1:09 pm

          This distinction is intentionnal. The way I see this is that AnimationClip/ AudioClip are the Asset (aka the Data) and the AnimationClipPlayable is the instance of the Asset.

          I think it makes total sense to have them as 2 different objects. For instance, you want to have 2 times the same AnimationClip in a graph but with different Time.

          Hope this helps clear things up.

        4. Thanks for the response.

          I see. If AnimationClip is an UnityEngine.Object which does neither process any update operation nor cache any current “playing” state data -which was created as such so it can be drag’n’drop’ed in the editor and assign to exposed fields- then I can understand the rationale (ditto for other classes like AudioClip), whether I like it, fully/partially, or not.

          However, the problem lays in the difference of design criteria between “native” (built-int) vs. “custom” (external) playables. The former a struct, the latter a class (which then has to encapsulate its instances within a containing ScriptablePlayable). This could bloat the system with corner cases and undesired results given the differences that stem from using a built-in or a custom playable: “I have created this playable class and there’s a spike on my perf graph when …”, “I have changed the value of X field on the AnimationClipPlayable which is already assigned to the graph but the latter is still using the old value”, and so on so forth.

          Again, I really think that this a great move towards the removal of legacy logic, but, and please take my words as a constructive criticism, I do believe the API needs a few more tweaks to avoid situations like the ones you may find when using: AnimatorController, AnimatorOverrideController, and UnityEditorInternal.AnimatorController (where modifying the clip assigned to a specific state on a layer at runtime outside the editor is NOT a trivial task).

  11. If the playable API is no more experimental, why the visualizer tool isn’t ship with unity directly, instead of having to download something to be able to create and debug more easily ?
    Are you planing to ship the tool in unity someday ?

    1. Pierre-Paul Giroux

      August 3, 2017 at 4:35 pm

      We are currently looking into making it simpler for our users to get their hands on C# source code like this.

      In 2017.2, we are introducing Unity Package Manager. This will be the first stepping stone for the system. We are using an incremental approach to integrate the system into the Unity ecosystem. It will grow with more features over time. For this first release, we avoid exposing user-facing features. Instead, we are exposing an API for enabling internal components to be updated more frequently than the editor (first pillar of the system).

      The graph visualizer will use this eventually, as will the Animation C# jobs samples.

      1. Thanks for the response.

        I see. If AnimationClip is an UnityEngine.Object which does neither process any update operation nor cache any current “playing” state data -which was created as such so it can be drag’n’drop’ed in the editor and assign to exposed fields- then I can understand the rationale (ditto for other classes like AudioClip), whether I like it, fully/partially, or not.

        However, the problem lays in the difference of design criteria between “native” (built-int) vs. “custom” (external) playables. The former a struct, the latter a class (which then has to encapsulate its instances within a containing ScriptablePlayable). This could bloat the system with corner cases and undesired results given the differences that stem from using a built-in or a custom playable: “I have created this playable class and there’s a spike on my perf graph when …”, “I have changed the value of X field on the AnimationClipPlayable which is already assigned to the graph but the latter is still using the old value”, and so on so forth.

        Again, I really think that this a great move towards the removal of legacy logic, but, and please take my words as a constructive criticism, I do believe the API needs a few more tweaks to avoid situations like the ones you may find when using: AnimatorController, AnimatorOverrideController, and UnityEditorInternal.AnimatorController (where modifying the clip assigned to a specific state on a layer at runtime outside the editor is NOT a trivial task).

        1. Sorry, this comment was meant for a different thread.

        2. Sorry. This response was meant for a different thread above.

  12. The wording “stream.human.IKSolve” makes it sound like this is only for humanoid characters. Does this mean that this playable API isn’t very useful for non humanoid rigs? Also how does this work with regards to 2D animation?

    1. Towards the end of the blog post, they say they are workig on a generic 2-bone IK solver example. There’s example code of it somewhere in the forums, but it’s not usable yet because it relies on the AnimationStream thingy.

      But in short; it’s coming

  13. PPG, it’s too much code to get what we used to have. Make an Animation component, at least so we can return to sanity.

    1. Pierre-Paul Giroux

      August 2, 2017 at 9:45 pm

      That is why we are doing a Legacy-like component. I agree it requires 2 components ( we are looking into maybe having component-less animation playback in the future.. but that is further down the line).

      cheers.

  14. I can’t read the article! :(

  15. Just dropping in to say this is awesome and we will always welcome more programmer/performance/versatility oriented features like this!

    I’ve been using the Playables API for a while now throughout its experimental phases and it’s been a saviour

    1. Good to hear ! Can’t wait to get feedback on this version!

  16. I have heard talk of playables for a while now, but still have no idea what they are or what it means. I feel like this has become an internal unity catch phrase and the rest of us aren’t quite sure what you’re taking about. Can someone please post some simple explanations and break this down to basics focusing on how the strengths of this workflow is actually improving on more traditional ones, thanks!

    1. Pierre-Paul Giroux

      August 2, 2017 at 7:09 pm

      The Playable API is, like its name says, an API. We are using this API in our AnimatorController and in our Timeline. The idea is too provide our users with a common API for different media type ( Animation, Audio, etc..) so that users can :

      1/ build their own tool that uses the API
      2/ use the API to do new things ( ie blend 2 AnimatorControllers together)
      3/ extend the functionally of current features ( ie you can put your own Playables in the Timeline… and eventually in the AnimatorController etc..)

      There is a link for the manual in the blog post which gives a lot of details on how this work and what it can done!

      Hope this clears things up!

    2. It’s actually pretty hard to explain clearly. I only started to understand their immense potential when I started playing around with them.

      This would be a great place to start:
      https://docs.unity3d.com/Manual/Playables.html
      https://docs.unity3d.com/Manual/Playables-Examples.html

      Unity advertises them as “a thing for animation, audio and video”, which could sound questionable and convoluted to certain people, but really it’s much more abstract and versatile than that. The playables API is a way to arrange “things” in a series of inputs and outputs, and you define exactly how you want your “things” blend together, and how you want to change your inputs. I won’t go any deeper than that in my explanation because I can tell right now that it’s way too abstract. Just look at the example

  17. Why the legacy-like example and how does this work with Mecanim? We moved away from legacy’s inflexibility for good reasons so a better example would be how this works with good tools like Mecanim. I see some use cases but I’m struggling to see practical application in normal everyday workflows and why I would want to use this for characters, or how I would even use it with Mecanim controllers.

    1. Pierre-Paul Giroux

      August 2, 2017 at 6:48 pm

      Hi.

      Something that we get requested a lot is the ability to play AnimationClip without the need of an AnimatorController. The legacy-like API provides this.

      We are also doing this for a couple reasons

      1/ give our users who still prefer a Legacy-like control (instead of our StateMachine), the ability to have a similar API but with the power of Mecanim animation system ( Retargetable, multhithreaded etc..)

      2/ give our users the possiblity to modify the legacy-like source code to suit their needs

      3/ proof of concept on the versatility of the Playable API

  18. Wendelin REICH

    August 2, 2017 at 3:35 pm

    Great stuff! Really looking forward to how this will allow more sophisticated procedural animation in Unity!

    For this to really take off, I think you should provide as much low-level infrastructure as possible. Especially IK solvers that work on any rig (not just humanoid) and that support both in-chain and end-of-chain constraints and effectors. And also a retargeting that isn’t limited to humanoid rigs.

    1. Pierre-Paul Giroux

      August 2, 2017 at 7:03 pm

      Yes.

      The C# animation jobs feature is still WIP we will post more on this in the future.

      The idea is that we want to provide a very very low level C# access to the Animation Stream. We will also provide source code for different uses cases ( IK, full body IK etc..) and you guys will be able to tweak and modify them!