Search Unity

Read how ADAM’s facial mocap and cloth simulation got into Unity, producing highly realistic CG humans, all rendered at 30 FPS in real time. To accomplish this, Neill Blomkamp and Oats Studios made great use of our Alembic Importer, Timeline, vertex sharing, and other standard features.

I’m SerEn (Sean) Low, an Art Tools Engineer specialized in Tools and Pipeline on the Made with Unity team. Prior to joining Unity Technologies, I was at Disney Interactive, developing animation tools for their console games, which collaborated with Pixar, Marvel and Lucasfilm. For the ADAM films, I helped integrate tools such as Alembic and Timeline, making sure the animation pipeline is streamlined and easy to maintain.

Background: The birth of Alembic in Unity

About two years ago, the Unity R&D team initiated the Alembic Importer plugin. It was used in a short film named The Gift by Marza Animation Planet. In one of the most challenging shots in the film, there are millions of plastic balls that form an ocean wave, which is done by importing the Alembic data generated from a Houdini simulation into Unity. It was a solid proof point that Alembic could be integrated into Unity by allowing artists to bring their simulated data from various external applications. Alembic also allows artists to break through the limitations of joint-based mesh deformation in a real-time engine.

When Oats Studios decided to use our Alembic Importer in their pipeline, they also wanted additional functionality and optimization so that it could handle high visual fidelity for cloth simulation and real-life facial-performance capture. Since the requirement was vastly different from what the Alembic Importer was originally developed for, Unity’s Dev team put in a huge effort to upgrade this plugin.

Major upgrades for the Alembic Importer

The first major update of the Alembic Importer was Timeline Integration. This allows Alembic data to stream in and playback through Timeline during runtime. Simultaneously, during edit time, the user can scrub through Timeline while viewing the Alembic playback. To enable this integration, we added a custom PlayableBehavior (Alembic Stream Player), PlayableAsset (Alembic Shot Asset) and TrackAsset (Alembic Track) to the Alembic Importer package. (For details about the Playables API, check out the documentation.)

In Timeline, you can then add an Alembic Track, create an Alembic Shot Asset and associate it with the Alembic instance in-scene with an Alembic Stream Player component attached.

An Alembic Track and Alembic Shot Asset in Timeline

An Alembic Stream Player component in scene object

The second significant update for the Importer was support for Vertex Sharing, which was important to optimize Alembic streaming performance for constant mesh topology assets. While typical Alembic data is used for dynamic simulation such as particles (where mesh topology is always changing), we wanted to merge shared vertices in order to optimize the playback performance. This is highly important since the mesh is reconstructed every frame based on the data streaming in real time. By merging the shared verts, other mesh properties such as edge, UV, normal and tangent are updated too, so the dynamic Alembic mesh now behaves like an FBX mesh. This ensures the mesh reacts according to assigned materials, shaders and lighting in a scene.

Alembic mesh in Unity showing shared vertices

For the third major update, we added Interpolation in Alembic vertex data. Since Alembic mesh is basically a new mesh constructed every frame, it doesn’t have the information about the delta of the previous and next vertex position. This is an issue when you try to apply motion blur onto Alembic mesh for rendering. With interpolation enabled, the Alembic asset in Unity is able to calculate the interpolation between positions and thus output the velocity that is necessary to calculate motion vectors. That means you can now have accurate motion vectors derived from the Alembic mesh to be used in shader computation or by the Post-Processing Stack. This enhances motion blur quality in final rendered images and it also improves performance because fewer samples are needed.

Motion vector on Alembic mesh

Before motion blur applied

After motion blur applied

The motion vector buffer

Additionally, several other production-friendly features were implemented to enable a smooth production workflow, such as auto-activate on Alembic Playable Asset, which enables the activation of the Alembic Playable Asset without the need of an additional Activation Track in Timeline. The Time Scale property allows you to re-time the playback rate of Alembic data. The Turn Quad Edge property allows you to choose which orientation the mesh faces should triangulate in. There is also an option to cache all Alembic samples whenever the Alembic scene object is enabled, instead of having to cache the data during playback.

Alembic in production at Oats Studios

“Alembic streaming was a very significant aspect of the project for us. Bringing simulated things over is difficult and the Alembic Importer in Unity is a game-changer,” said Oats Studios’ VFX Supervisor, Chris Harvey.

Assembling multiple Alembic assets in Timeline editor

Based on real wardrobes, all the clothing in Oats’ ADAM films was captured by photogrammetry and then recreated and simulated in Marvelous Designer, where digital tailor Sean Frandsen applied real physical attributes and animation to the different fabrics (wool, cotton, leather, etc.).

Meanwhile, facial performance was captured and converted into a series of static meshes for Alembic playback. (For more information about facial capture, watch the ADAM: Behind the Scenes video.)

In ADAM, Alembic is mainly used for two types of assets: Cloth and faces

The data was then imported into Autodesk Maya for further cleanup and tweaking before being exported out as an Alembic file. Here’s a schematic of their pipeline.

 

The Alembic and FBX assets pipeline into Unity

Assembling simulated assets in space and time

At this stage in Unity, Oats used Timeline as an assembly tool, combining Alembic assets and choreographing them in scene space along with other assets. For example, in order to fully assemble a full character performance, they used two Alembic tracks, one each for facial and cloth, on top of an Animation track, which contains body performance in FBX format as shown below.

The assembled character comes to life in Timeline

With everything in Timeline, Oats could synchronize and sequence all asset performances according to each shot’s timing for real-time playback. However, large numbers of assets can easily overcomplicate the Timeline editor interface and be hard to maintain once in production. Oats solved this by organizing assets in-scene using TrackGroup, which also let them expand or collapse groups as needed. Another Unity feature that Oats tapped to keep things well organized and manageable was ControlTrack for handling multiple nested Timeline instances.

Expanded and collapsed Track Groups

ControlTracks handle nested timeline assets (Characters and Caches shown here)

One-click automation

Later in the production stage, the amount of generated Alembic data for both facial mocap and cloth simulation reached almost 300 files. Therefore Oats needed an automated process to import multiple Alembic files into Timeline, so they implemented their own importer to automatically bring in Alembic files with predefined settings, instantiate them into a scene with the required components as well as sequence them in Timeline. With help from the Unity Dev team, Oats was able to kick-off this automation process with the click of a button. We also gathered a lot of great feedback and API requests from this project to further improve Timeline.

Managing hundreds of Alembic assets for various characters and props

Looking ahead

Over the course of the ADAM project, the Alembic Importer became more mature and production-ready. You can download the release version at the quick link below. Now you can use it for dynamic topology simulation mesh as well as static topology mesh such as cloth and face. It’s optimized and has better integration for existing and upcoming Unity features. More and more artists and studios are showing interest in using Alembic, especially for real-time CG films, so we’re looking forward to seeing how it will enable the success of many awesome Unity projects. Let us know how you’re using it!

Special thanks to the Unity DCC Dev Team: Mark Visser, Jason McCullough, Thomas Chollet, Seiya Ishibashi

More “Behind the Scenes” in ADAM
Check out our other in-depth posts on lighting, clothing simulation, and Timeline.

Quick link

You can get Alembic Vertex Sharing release version (1.0f1) in Unity Package format here.

(This branch will be merging with the master soon. We will update again once it’s ready. Sorry for the confusion caused.)

Learn more about Unity

Find out how Unity 2017 and its features like Timeline, Cinemachine, Post-Processing Stack, and real-time rendering at 30 FPS help teams like Oats Studios change the future of filmmaking.

 

 

 

42 评论

订阅评论

评论被关闭。

  1. Eddie Christian

    二月 1, 2018 2:45 上午

    So after testing this I have to ask. Why is the version on GitHub so far behind what has been done with the Adam Series. None of these shorts could have been made with this Plug-In in it’s current state. Does Oats Studio have a Dramatically different version that we could have access to?

    1. Hi Eddie,
      We were using the vertex-sharing branch for Adam project, that’s might be why you see the main line is behind. It’s been merged now and we’re doing all necessary tests. So no, Oats doesn’t have any special build, the Alembic Importer is publicly available :)

  2. Eddie Christian

    一月 9, 2018 9:26 下午

    According to Sean in the below quote Now this is integrated into Unity. But Import Asset does not even see .abc files…

    “Sean Low
    December 11, 2017 at 7:46 pm

    Hi Narendra,
    Alembic Importer is now integrated so you can go Asset>Import New Asset and point to a .abc file. You will see the abc file asset in your Asset folder :) You can then drag drop the abc into your scene. To playback the alembic:
    1. Hit Play, and the alembic stream will kick in.
    2. In Timeline, add Alembic track, add Alembic clip, then link the abc scene obj in the ‘Stream Player’ of Alembic Shot Asset.
    Hope helps :)”

    1. Hi Eddie,
      You will still need to download the Alembic Importer from the link mentioned in the article. Once it’s installed, you’re good to go! :)

  3. Really great tool!
    I am importing alembic files to use in timeline for a 360 rendered movie. The reason for alembic is the mesh is changing poly count over the sequence. Think lava lamp / blob mesh. Some imports with a larger surface area are divided into smaller chunks on import. This works perfectly if the material is opaque but not if the material is transparent. lots of flashing faces etc. Is it posibble for somehow force the import to keep the mesh as one on import.

    1. Hi Ca935,
      This ‘vertex-sharing’ Alembic Importer is meant for static/ constant vert count mesh, eg: face, cloth. We will keep your feedback in mind and hopefully to update more on that :)

  4. Can we get a bit more info on what’s being updated by the interpolator? I assume vertex data and possibly normals, but what about UV?

    I ask because I have an alembic clip with a sample rate of 5 frames. When it’s animating in Unity, it loops choppy with “interpolate samples” turned off (as expected), but is smooth when “interpolate samples” is enabled. However, the shading (shadows) on it looks pretty bad, and I’m trying to figure out why and how to fix this.

    1. Hi BlackPete,
      A simple repro would be super helpful for us to figure out more :)

  5. Works great but I am not seeing the Material assignment to faces. Just brought in a character from Maya 2017. It comes in with just one material.

  6. I have created 10 animations in Unity about the Ten Hox Herding Pictures .
    https://youtu.be/DDZFfgq3e1s?list=PL3Fu_BwnEqetg6Z20t_G9TgLY6-srxqEG

  7. Hi Sean Low,
    Thanks for your tool which will open news doors for artists.
    I am working on a VR project in UE4 to create a tool which will allow the artist and the customer to see the progression of the work.
    https://youtu.be/7qWXEylM3ns
    I was working with an Htc vice but recently I acquired an Oculus Rift and start to work with Unity.
    I am very dependent of Alembic as I import clothes from Marvelous.
    My question is: Is it possible to import an .abc file from a folder outside the builded game?
    Because as this is a tool the artist must be able to import his/her creation inside.
    Thanks for your cooperation.

    1. Hi Heurtaux,
      Thanks for sharing your VR project!
      You can import .abc file from any location and the Importer will create a copy in ‘StreamingAssets’ in your Unity project, to be used when you build your game.

  8. I think I’m still confused by how the new importer works.

    So I drag in an abc file into the Assets folder. A copy of this is made in StreamingAssets.

    Is there a way to prevent having two copies of the abc file? That can really eat up HDD space if the abc files are multi-gigabytes in size…

    1. Hi BlackPete,
      Yes, Alembic Importer will create another copy in StreamingAssets which will be used when publishing the project. In ADAM films, most of the single abc file are within MB, and total abc files is less than 10 GB . If you are doing mesh deformation like a character, full body cache is not recommended. Try split it and only use Alembic where you need high detail deformation. Hope helps :)

  9. Hi,

    How do you import Alembic file to the scene? In the previous version, there used to be Alembic Import Window. I can’t seem to find that now.

    I was able to add Alembic timeline track but that’s all I can do with this package. Can you guys help me out?

    Thnx ^-^

    1. so it creates alembic prefabs if you copy into Assets folder (and NOT in StreamingAssets folder).

      SORTED! ;D

      1. Glad to know it works for you now! Feel free to ping us again if you have any question or share your works :)

  10. Wow, I’ve been using Alembic importer for my project at http://hrzn.org while it was at the beginning of its path, I am pretty happy about the optimizations, since .abc files can weigh a lot!

    1. Very cool! Feel free to let us know if you have any question or feedback.

  11. I would really like to know more about UDIM enchancements to the engine and whether / how we will use them in games. I hate UV system, UDIM is so much better and easier to create.

    1. You can check out this article, see if it helps. Also, don’t forget about our forum and asset store :)
      https://blogs.unity3d.com/2017/08/01/multi-material-using-custom-inspectors-and-scriptable-objects-for-udim-materials/

  12. Is the Github link for Alembic link the latest one? https://github.com/unity3d-jp/AlembicImporter

    It hasn’t been updated for 7 months and still uses the old Japanese examples.

    1. @Narenda – you’ll want to switch to the vertex-sharing branch: https://github.com/unity3d-jp/AlembicImporter/commits/vertex-sharing
      The last commit was Nov. 30th

      1. Nice!! this is what I wanted. Thnx

      2. Thank you Amir! :)

    2. I’ve added the vertex-sharing branch at the end of the article. Sorry for trouble caused! Please let me know if you have any question getting it in your Unity project.

      1. Hey Sean,

        so dll files seem to be missing https://github.com/unity3d-jp/AlembicImporter/tree/vertex-sharing/AlembicImporter/Assets/UTJ/Alembic/Plugins/x86

        I tried to compile via cmake but it doesn’t compile because of missing glew library. Can you add dll files to plugin directory?

        Thnx

        1. Sorry again. We now have Unity Package released version for easier direct download. Updated the link above, thanks again for your feedback :)

      2. Hey Sean,

        How do you import Alembic file? There used to be an alembic import window in the previous versions. I can’t seem to find it in the Menus.

        Thnx

        1. Hi Narendra,
          Alembic Importer is now integrated so you can go Asset>Import New Asset and point to a .abc file. You will see the abc file asset in your Asset folder :) You can then drag drop the abc into your scene. To playback the alembic:
          1. Hit Play, and the alembic stream will kick in.
          2. In Timeline, add Alembic track, add Alembic clip, then link the abc scene obj in the ‘Stream Player’ of Alembic Shot Asset.
          Hope helps :)

  13. Good News! Do you have any plans adding Android-Support for this plugin? I think for now you just can play Alembic animations on window, linux etc. Would be nice to have support for mobile devices too.

    1. Thanks for your interest! You’re right, Alembic support is currently only on desktop. Mobile support is being reviewed by our DCC team. We will updates again! :)

  14. This is just fantastic. Especially for making all this accessible.
    Kudos.
    b

    1. Glad to hear that! This is align with Unity’s mission: Democratize Game Development – Solve Hard Problems – Enable Developer Success :)

  15. Thanks for great detailed post

    1. You’re welcome! Hope it helps :)

  16. Please, please make the skin shader available as asset!

    1. Go check scriptable render pipeline. it’s there

      1. Can you give us a link? can’t find it…

    2. The skin shader in this project is soley on our legacy render pipeline, as ADAM EP2 & EP3 were built on 2017.1. My teammate will be posting an article about the shading part in this project soon. Please keep an eye on that!

  17. This blogpost is a great culmination of Oats’ and Unity’s efforts with Adam,
    Interweb thanks and hopes there are more to come.

    1. You bet! We just released another article about Timeline in ADAM films. Hope you will like it too :)