Search Unity

You can improve the runtime efficiency of your shader without sacrificing the quality of your graphics by packing physically based rendering (PBR) material information into a single texture map and layering it into a compact shader. Check out this experiment.

This experiment works in both the Universal Render Pipeline (URP) and High Definition Render Pipeline (HDRP). To get the most out of this article, you should have some familiarity with Shader Graph. If you are new to Shader Graph, please explore our resources for an introduction and more detail about this tool for authoring shaders visually.

When working with art assets in a terrain-like environment, multiple layers of tileable material are typically preferred as they produce better blending results. However, the GPU performance cost of multiple texture samples and growth of memory use with each layer added to the shader can be prohibitive for some devices and inefficient in general.

With this experiment, I aimed to:

  • Do more with less
  • Minimize the memory footprint and be frugal with texture sampling in representing a PBR material
  • Minimize shader instructions
  • Perform layer blending with minimum splat map/vertex color channels
  • Extend the functionality of splat map/vertex color for extra bells and whistles

 

While the experiment achieved its goals, it comes with some caveats. You’ll have to set your priorities according to the demands of your own project in determining which trade-offs are acceptable to you.

 

1. Single texture-packed PBR material

Before layering, the first thing you need to do is figure out the PBR material packing. PBR material typically comes with the parameters for Albedo (BaseColor), Smoothness mask, Ambient Occlusion, Metalness, and Normal defined.

Usually, all five maps are represented in three texture maps. To minimize texture usage, I decided to sacrifice Metalness and Ambient Occlusion for this experiment.


The remaining maps – Albedo, Smoothness and Normal Definition – would traditionally be represented by at least two texture maps. Reducing it to a single map requires some preprocessing of each individual channel.


The final result of the PBR Material packed into a single texture.

Red = dHdu (Derivatives Height Relative to the U direction) for Normal Definition#.
Green = dHdv (Derivatives Height Relative to the V direction) for Normal Definition#.
Blue = Linear Grayscale shade representing Albedo (color reconstructed in shader).
Alpha = Linear Smoothness map (standard Smoothness map).

Note: The texture is imported into Unity with sRGB unchecked and compressed with BC7 format. When porting to other platforms, switch to the platform-supported equivalent 4-channel texture format.

Processing the maps

Albedo

Albedo is normally defined as an RGB texture; however, many terrain-like materials (rock, sand, mud, grass, etc.) consist of a limited color palette. You can exploit this property by storing Albedo as a grayscale gradient and then color remapping it in the shader.

There is no set method for converting the RGB albedo to a grayscale gradient. For this experiment, The grayscale Albedo was created through selective masking of the original Albedo map channels and Ambient occlusion; to match the prominent color in the shader color reconstruction, just eyeball any manual adjustments.

Highlight and Dark coloring with ramp contrast adjustment for Albedo reconstruction.


Smoothness

Smoothness is considered very important for PBR material definition. To define smoothness more precisely, it has its own channel.

A simple multiplier was added to the smoothness in the shader for some variation in the material.
(You can choose to skip smoothness multiplier to reduce shader instruction.)


Normal definition

The Normal map is important for showing the detailed characteristics of a surface. A typical PBR Material uses a tangent space normal map. In this experiment, I chose a pre-converted derivatives map using surface gradient framework for the reasons below. (See Morten Mikkelsen’s surface gradient framework for more information).

To pre-convert tangent space normal maps to derivatives, use this Photoshop action.

Using a pre-converted Derivatives map has several advantages:

  • Can be directly converted to surface gradient, using fewer instructions than a standard tangent space normal map, which requires derivatives conversion in the shader
  • Can be stored in two channels (dHdu and dHdv), resulting in a lower memory and texture cache footprint in runtime 
  • Does not require blue channel reconstruction in the shader, which is typical when processing tangent space normal maps, since the surface gradient framework takes care of the normal reconstruction (fewer shader instructions)
  • Works correctly when adjusted in Photoshop – that is, by blending, masking or reducing intensity – and does not require renormalization. For example, to reduce intensity, simply blend the map against RGB(128,128,0).

In conjunction with the surface gradient framework, the advantages further include:

  • Normal bump information can be blended and composited in the shader the same way as albedo blend/composite, with the correct result.
  • Increasing, reducing and reversing bump contributions is trivial and accurate.

But pre-converted derivatives from tangent space normal map also have some disadvantages:

  • Using Photoshop conversion, normal definition gets clamped at an angle greater than 45 degrees, to balance precision in an 8-bit texture.
  • Artists are used to working with tangent space normal maps and require the maps to be pre-converted via Photoshop as part of their workflow.

Note: Clamping at an angle greater than 45 degrees does not apply to shader-based derivatives conversion.

Depending on your use case, the limitation may have a lesser or greater effect. In this experiment, a normal direction less than 45 degrees does not have a noticeable negative impact on the end result. In fact, in this case, it provides a benefit by reducing unwanted reflection from an extreme normal direction.

A comparison of a pre-converted derivatives map texture and a tangent space normal map texture.

 

Stress test differences between pre-converted derivatives and tangent space normal map. Notice that the Spherical bump exhibits some artifact due to the 45-degree clamping in the pre-converted derivatives map.

 

The graph to remap the pre-converted derivatives map to surface gradient.

 

Note: Make sure the Master stack fragment normal space accepts world space normal instead of the default tangent space normal.


The full unpacking process 

The complete Sub Graph to unpack the Compact PBR texture to output colored Albedo, smoothness and surface gradient.

Note: Surface gradient conversion to Normal is done outside the Sub Graph so that the material can be easily blended based on the output of the UnpackedSubGraph.

The materials sample from the unpacked shader. 


Different angles to better showcase each material’s responses to light.


2. Layering the material

For this experiment, I chose a tier-based layering method on a single channel remap. The Sub Graph does five linear interpolations (plus the base, forming six layers).

There are many ways to blend layer weights. This method has the simplicity of a single vector input, which suits the experiment goal. This allowed lots of layering without burning through multiple channels in splat maps or vertex channels.


The drawback of this method is that you cannot control the weight of an individual layer’s contribution. The blending will always be a transition from the previous layer. Depending on the use case, this can be a limiting factor compared to a traditional per-channel blend.

 

The Sub Graph to remap a single channel to represent the six layers.

 

The Sub Graph shown above is predefined for six layers of tier-based blending. To create more layers, divide 1 by the desired number of layers blended, subtract 1, and then remap each layer based on that value range.

For example, for a nine-layer blend material, each layer remap range is 1/(9-1) = 0.125.

Layer In Min In Max
1 0 0.125
2 0.125 .25
3 .25 .375
4 .375 .5
5 .5 .625
6 .625 .75
7 .75 .875
8 .875 1

Be aware that as you divide the single channel into smaller portions, you have less shading range.

The layering method tested using color to showcase the blending characteristic.

 

The material to composite the 6 layers of the unpacked material with a single vertex color (in this case, the red channel).

Note: It’s simpler to blend the layers of a single sample-packed PBR material in the Shader Graph than to blend three PBR textures.

Since layer blending is tier-based, it’s a good idea to plan the order of the layers before you set up the blend material.


3. Extending the functionality of material

Layer blending requires only a single channel (the red vertex channel). The remaining three vertex channels offer extra functionalities.

The final Shader Graph produces results using the remaining vertex channels.

In this experiment, vertex painting was done inside Unity Editor using Polybrush (available from the Package Manager).

Suggested Vertex Paint color palette for this shader.

 

Red: Used to weight the layer contribution.

Red vertex channel painting demo


Green: Sets the surface gradient property, to flip, reduce or add normal bump contribution (remapped to -1 and 1).

  • 0 reverses the normal bump (-1)
  • 0.5 value zeroes out the normal bump (0)
  • 1 sets the normal bump to the original value (+1).

Green vertex channel painting demo

 

Blue: Controls smoothness and surface gradient bump scale to create a wet water look

  • 0 = no alteration
  • 255 = maximum smoothness and flat normal map (wet look)

Blue vertex channel painting demo

 

Alpha: Controls the weight of the Albedo layer, setting the base color to white, with the contribution based on the y axis of the surface normal. It does not alter the smoothness and takes advantage of the original surface layer smoothness and bump property.

  • 0 = no snow
  • 255 = solid snow

Alpha vertex channel painting combined with previous channels to showcase how the whole layers interact with the snow

The combined results of the different vertex painting channels:

You can adjust the shader blending method and the settings for the various vertex channel/splat map functionalities according to your project’s requirements.


4. Comparing performance

The purpose of this experiment was to extend the functionality of the Shader Graph while minimizing resources. The texture was preprocessed and unpacked, but is there a payoff in runtime efficiency?

Performance profiling shows the efficiencies these efforts produced. 

A standard six-layer blend shader was created for comparison with the compact six-layer blend shader. Both shaders were created using an identical blending method with the same functionalities. The main difference is that the standard shader uses three different textures to represent a single layer.

For profiling, a single mesh was rendered on screen with blend material using the Universal Render Pipeline in the targeted platform.


Mobile memory and performance profile

Texture compression for mobile (Android):

Standard PBR with Albedo, Mask and Normal map at 1024×1024 for mobile:

  • 6x Albedo map ASTC 10×10 = 6x 222.4 KB
  • 6x Mask map ASTC 8×8 = 6x 341.4 KB
  • 6x Normal map ASTC 8×8 = 6x 341.4 KB

Total Texture memory usage 5.431 MB

 

Compact PBR at 1024×1024 for mobile:

  • 6x PackedPBR Texture ASTC 8×8 = 6x 341.4 KB 

Total Texture memory usage 2.048 MB

 

With the compact six-layer material, there is approximately 62% Less texture memory consumption on Mobile (Android), savings of more than half.


Mobile Android/Vulcan with Adreno 630 (Snapdragon 845);
Snapdragon profile results:

  • Approximately 70% less texture memory read in runtime.
  • Standard took 9971020 clocks to render.
  • Compact took 6951439 clocks to render.

Compact material renders on screen approximately 30% faster.


Profiling result from Snapdragon Profiler.


PC memory and performance profile

Standard PBR with Albedo, Mask and Normal map at 1024×1024:

  • 6x Albedo map DTX1 = 6x 0.7 MB
  • 6x Mask map DXT5/BC7 = 6x 1.3 MB
  • 6x Normal map DXT5/BC7 = 6x 1.3 MB
    Total Texture memory usage 19.8 MB

 

Compact PBR at 1024×1024:

  • 6x PackedPBR Texture BC7 = 6x 1.3 MB 

Total Texture memory usage 7.8 MB 

 

The compact six-layer material uses 60% less texture memory consumption on PC (savings of more than half).

 

PC laptop with Radeon 460 Pro rendering at 2880×1800; RenderDoc profile results:

  • Draw Opaques for standard 6-layer blend: 5.186 ms.
  • Draw Opaques for compact 6-layer blend: 3.632 ms.
    Compact material renders on screen approximately 30%* faster.
    *RenderDoc profile value fluctuates; 30% is an average of samples.

 

PC desktop with nVidia GTX 1080 rendering at 2560×1440; nSight profile results:

  • Render Opaques for standard 6-layer material: 0.87 ms
  • Render Opaques for compact 6-layer material: 0.48 ms

Compact material renders on screen approximately 45% faster.


Profiling results from nSight.

 

Console performance profile

On PlayStation 4, using compact material yields 60% memory savings, identical to that for PC as the PS4 uses the same compression.

 

PS4 base rendering at 1920x 1080; Razer profile results:

  • Render Opaques for standard 6-layer material: 2.11 ms
  • Render Opaques for compact 6-layer material: 1.59 ms
  • Compact material renders on-screen approximately 24.5% faster.

Profiling result from PS4 Razor profiler.


In summary, using a compact six-layer PBR shader offers performance gain and significant memory savings. The variation of GPU performance is interesting but expected, as unpacking the material consumes more ALUs than sampling more textures.

Download the sample project

This sample project with Shader Graphs and Sub Graphs can be downloaded here:

[DOWNLOAD HERE], Unity 2020.2.5f1 with HDRP 10.3.1
[DOWNLOAD HERE], Unity 2020.2.5f1 with URP 10.3.
[DOWNLOAD HERE], Photoshop action to pre-convert tangent space normal map to derivatives.


Takeaway

Screenshot from Universal Render Pipeline version of the project.

The main components of this experiment are:

  • Shader Graph for custom material
  • Pre-converted Derivatives
  • Surface gradient framework
  • Albedo color reconstruction
  • Single-channel layer blending
  • UpVector blend technique, smoothness and bump control via vertex channel blend

This experiment showcases how you can use Shader Graph to produce beautiful graphics that are also efficient. Hopefully, this example can inspire artists and developers to push aesthetic boundaries with their Unity projects.

 

About the author

Rinaldo Tjan (Technical Art Director, R&D, Spotlight Team) is a real-time 3D artist with an extreme passion for real-time lighting and rendering systems.

Having started his career in the PlayStation 2 days, he has more than a decade of end-to-end artist workflow knowledge, from texturing to final rendered scene creation. Prior to joining Unity Technologies, he helped deliver AAA games such as BioShock 2, The Bureau: XCOM Declassified, and Mafia III.

He currently works with Unity clients to help them augment their projects and realize their true potential using Unity while helping drive the internal development and standards of Unity rendering features.

21 replies on “Experimenting with Shader Graph: Doing more with less”

Hello! great job!
I am really curious to adding this some real time mesh gen for close up camera.
I noticed that this terrain is realistic from far but not from close up camera.

I want to make a tech that converts these texture 3d effect to convert 3d high res vertex like tessellation when player look in front of terrain . So then whole world becomes Realistic!

For the moment, there’s no timeline I can give you for the tessellation shader graph functionality.
However there’s definitely a few things you can try to experiment with your idea.

For the vertex displacement, it is already supported at the moment with Shader Graph.
Just need to apply it to a highly tessellated mesh.
Then to test it even further, use the LOD Group system that has multiple progression of polygon complexity on the mesh to see if the result is as desired.
While this doesn’t give you the smooth transition of the auto tessellation system, it gives you a general idea on how the 3D displacement will look like.

Looking at your video, you might be interested in looking at Gradient Map technique.
You can try this in Shader graph.

Connect Sample 2D of a Grayscale image (R) to -> (Time) input of «Sample Gradient» node with a Gradient property connected to it’s Gradient input -> output of the sample gradient to base color to see the results.

This technique is known in compositing software and Photoshop (I believe GIMP has similar functionality). That way you can preview how the gradient get’s mapped to color the grayscale images.
Give it a shot.

Good article but relying on photoshop actions is a negative point.

It would be better to do a quick shader inside Unity to do the same result to process textures. It’s more reliable this way, can easily be absorbed into existing toolchains and can be saved as you’d like inside Unity.

I do this for my own conversions and so on, plus most people do not have an adobe subscription, only professionals tend to.

Feedback taken. The same concern was posted by mylk below in comment section. In hindsight I should have included the shader sub-graph (or at least a screenshot) to show what the ts normal to derivatives action try to accomplish. For now, please follow the link I posted in mylk reply to get the subgraph.
In any case, thank you for taking the time to read the blog and posting feedback.

No problem, I felt the article was one of the best I’d ever seen in recent times from Unity, and hope there will be more. I could only find one minor problem as above :-)

Nice writeup!

A few thoughts:
– Texture Compression favors the green channel, and packs alpha separately. So your most important information should be in the green and alpha channels, with the least important in blue.
– Splat map texturing is almost always memory bound – in most cases, the cost difference of avoiding a few instructions is negligible, so focusing on better packing of textures is the way to go.
– Consider generating your smoothness/ao from your luminance and normal, with user controls to range map the smoothness. You should be able to get a good approximation from both with only a few instructions.
– In the Walking Dead game I did for mobile, we reduced our terrain to a single texture sample. We packed 4 luminance channels into a single texture, then stored 4 Cb and Cr in the vertices (yCbCr color space). We then generated a height value from the luminance for height blending, unpacked the 4 colors, and blended. We also produced a simple normal from the luminance, though for more modern devices I’d like just use the derivatives of the worldPos to get a decent approximation if I didn’t need to get close to the terrain.
– Being able to properly branch in the shader and use dynamic flow control would have seen a similar reduction in memory bandwidth usage. It’s sad that no shader graph I’ve seen can handle this properly. You would have hit six samples without any packing of textures, and 2 with packing.
– Given that your shader only ever uses 2 of the samples on any given pixel (due to the 1 channel splat map), you should never have to sample all 6 textures. You could achieve this in two ways- the first is to use a texture array to store your textures and determine which two texture indexes need to be sampled for the pixel, and only sample those. The second would be to branch around the unused textures, which unfortunately requires proper dynamic flow control.
– Note that getting down to two samples and using dynamic flow control, you’d be able to add stochastic height map sampling or triplanar texturing to this and have a similar cost to what you have now, both of which greatly help with the quality of terrain texturing. Properly branch sampled, these would be 4-6 samples per pixel using the combined approaches.

Really nice to get some insights from real world experience.
There are definitely alternatives trade-offs that can be made.
One of the priority in my experiment is to preserve the smoothness information as best as possible.
It would be interesting to experiment with what you suggest on generating smoothness/ao from luminance and normal, I have never tried that.

Is it possible to share all the needed actions to pre-convert tangent space normal maps to derivatives, for those who don’t use Photoshop?

Deja una respuesta

Tu dirección de correo electrónico no será publicada. Los campos obligatorios están marcados con *