Search Unity

When planning The Blacksmith short film, we never really prioritized a custom skin shader high enough for it to have any realistic chance of being picked up as a task. However, we still wanted to see if there was something simple we could do to add a little extra life to the Challenger’s expressions. After a quick brainstorming, we decided to have a go at adding blendshape-driven wrinkle maps to the project.

To add detail and depth to the expressions, we decided that the Standard shader would give us the best bang for our bucks if we let the wrinkles affect both normals and occlusion. We also wanted a method of restricting the influence of certain expressions to specific parts of the face.

Enter Wrinkle Maps Driver

We created a component that allowed the animator to define the wrinkle layers, one layer per blendshape in the mesh. The layer definitions contained texture mappings and strength modifiers, as well as a set of masking weights that would be matched against a face part masking texture. Using the masking weights, specific wrinkle layers could affect one-to-four of the masked face parts, each with a different influence.

challenger_facemask_inspector

Since we wanted to be able to blend up to four different expressions at any given time, the blending alone required 11 texture samplers with all bells and whistles enabled (two base textures, eight detail textures and one masking texture). The only realistic option for this was to compose the blended wrinkle maps in an off-screen pre-render pass. We found that the ARGB2101010 render texture format was perfect for us, as it would allow us to pack normals into two of the 10-bit channels, with the remaining one receiving the occlusion. Each frame, the wrinkle map component would find the four most influential blendshapes, and assign layer rendering weights accordingly.

challenger_screenspace2

Once we had all the wrinkle data composed in screen-space, the only remaining thing to do was to redirect the normal and occlusion data inputs in the Standard shader we were using for face rendering. In practice, this just meant adding a handful of lines to the surface shader main function.


// Sample occlusion and normals from screen-space buffer when wrinkle maps are active
#ifdef WRINKLE_MAPS
float3 normalOcclusion =
tex2D(_NormalAndOcclusion, IN.screenPos.xy / IN.screenPos.w).rgb;
o.Occlusion = normalOcclusion.r;
#ifdef _NORMALMAP
o.Normal.xy = normalOcclusion.gb * 2.f - 1.f;
o.Normal.z = sqrt(saturate(1.f - dot(o.Normal.xy, o.Normal.xy)));
#endif
#endif

Final Results

Comparing the base head to the – exaggerated – angry blendshape at full weight illustrates the additional detail added in by the blended wrinkle maps:

challenger_nowrinkles3

challenger_wrinkles3

We also added various debug output modes that allowed us to easily visualize the fully blended occlusion and normal maps. These were quite useful in figuring out exactly which component contributed to what in the final result.

challenger_debugviews

Take it for a spin

We’ve broken this feature out into an example project which you can get from the Asset Store. It’s basically just the Challenger’s head with a couple of the expressions we used in The Blacksmith, but should serve as a useful starting point for getting this system running in your own projects.