Search Unity

Faster builds with the new Caching Shader Preprocessor (Experimental)

, maio 6, 2020

Shader compilation is a lengthy process that involves several steps. One of the first steps is preprocessing the shader source. Unity 2020.1 beta introduces the Caching Shader Preprocessor, a new custom solution to perform this compilation step. It is up to 25% faster than the platform compilers’ preprocessor, closely follows the C standard, and offers new features.

Anyone who uses Unity deals with shaders, directly or indirectly. Shaders are programs that run on the GPU. They determine how objects are rendered to the screen. When, for example, a point light is added to a scene, objects that need to react to it will use a different shader variant, which defines how to handle point lights.

Whenever a new shader variant is used that has not yet been encountered, the Unity Editor has to compile it. When building the application to run on your target device, the Editor needs to compile all of the required shaders.

A checkbox to switch the Caching Shader Preprocessor on or off can be found in Editor settings in the Shader Compilation section.

The rest of this blog post covers technical details that you may find useful if you are writing shaders.

Built for speed

The new preprocessor caches intermediate preprocessing data to speed up shader import and compilation time. With this caching, the Editor doesn’t need to parse include files until their contents change, and compiling multiple variants of the same shader is more efficient. Enabling the new preprocessor has the most noticeable effect when shaders within a project use a large set of common include files.

Unification

Different shader compiler back ends come with their own preprocessors. Running a custom standardized preprocessing layer before the compiler is executed ensures that shader code is compatible with any compiler back end used currently or in the future, which in turn means easier project upgrades between Unity versions.

The same preprocessor is now used during shader importing, replacing the plain text parsing solution employed previously for checking shader compilation parameters (#pragma directives). This enables us to offer additional new features, such as conditional #pragma directive selection and a possibility to put those directives in include files.

New features

Show preprocessed source
Shader inspector now has a “Preprocess only” checkbox so you can toggle between showing compiled shader code and preprocessed shader source. This feature is very useful for anyone developing shaders and facilitates shader debugging.

Shader compilation parameters in include files
We introduced a new preprocessor directive, #include_with_pragmas, that allows shader developers to put #pragma directives that control compilation settings in include files. They act as regular #include directives but are not ignored during shader import time.

This distinction minimizes shader import time; otherwise, we would need to run the preprocessor on all the files included in the shader.

The old shader import path ignores the new directive.

Conditional shader compilation directive selection
Running the preprocessor during shader importing makes it possible to wrap #pragma directives that control shader compilation in preprocessor conditionals.

Not every conditional can affect #pragma selection. The preprocessor detects where the arguments for the conditional expression come from. This prevents recursion and minimizes intrusion into the shader compilation pipeline. The conditional affects #pragma directive reporting only if it depends on generic macro definitions (Unity version or UNITY_OLD_PREPROCESSOR macros), user macro definitions, or platform keywords that depend on the build target and cannot be configured by other means. The latter include SHADER_API_DESKTOP, SHADER_API_MOBILE, UNITY_NO_CUBEMAP_ARRAY, UNITY_FRAMEBUFFER_FETCH_AVAILABLE, UNITY_USE_NATIVE_HDR and UNITY_NO_RGBM as well as console SHADER_API_<CONSOLE> keywords, because these depend on build target only. Note that user macro definitions do not include user keywords specified with #pragma multi_compile or #pragma shader_feature directives.

Example of valid usage:

Example of invalid usage:


Changes in preprocessing behavior
The new shader preprocessing solution features the following noteworthy differences in behavior:

  • #pragma warning is now supported
  • #include directive parameters are now macro-expanded if they don’t match the regular format (“#include <my_include_file>” or “#include “my_include_file””)
  • #line directive parameters are now macro-expanded if they don’t match the regular format (“#line” followed by unsigned integer and, optionally, by a file name)
  • When a macro is redefined, a warning is issued if the new definition differs from the previous one
  • Macro expansion fully conforms to the C language standard
  • Ill-formed #undef directives are no longer ignored if the first argument is an identifier
  • Stringizing is now supported fully
  • Concatenation now works in macros without arguments
  • Invalid concatenation now issues a warning
  • Macro parameters are not expanded when used for stringizing or concatenation
  • Shift operators are now supported in conditional expressions
  • When the preprocessor encounters a macro name that has no parameters when it should have parameters, no error is generated and no macro expansion happens
  • “\\\n” (backslash followed by newline) is not treated as space anymore

The old shader import pipeline now defines a UNITY_OLD_PREPROCESSOR macro, which can be used to distinguish between the two preprocessors.

Let us know what you think

Caching Shader Preprocessor is a new feature. It is marked as experimental in Unity 2020.1 and applies across all platforms and render pipelines. 

We intend to have it fully available by the end of the 2020 version cycle, so try out the feature in Unity 2020.1 beta and give your feedback on the forum.

 

1 reply on “Faster builds with the new Caching Shader Preprocessor (Experimental)”

Thanks

For me the most time consuming part is the texture compression. It takes me a day on my 100gb project
I hope this can also be accelerated at some point.

Comments are closed.