Light & Shadows, based in Paris, has used HDRP to produce stunningly realistic real-time images and videos. This blog post from Light & Shadows walks through the processes they’ve used to achieve these excellent results.
History of Light and Shadows
Unity has worked closely with Light & Shadows on projects to demonstrate lifelike visual quality in real time. This blog post has been written by Light & Shadows to provide technical insight into how they used Unity’s new HD Render Pipeline to produce an amazingly realistic video of the Lexus LC 500 in real time. Light & Shadows was founded in 2009 in response to growing demand by major industrial companies for compelling visual content. From its founding, the company has thrived by continuously adapting and innovating the delivery of new capabilities to its customers, including the generation of high-quality rendering together with cost reduction and productivity improvement. The rest of this blog post is from Light & Shadows.
Real-time rendering: A game changer for the automotive industry
Technology is transforming the way we experience, sell and buy cars. Light and Shadows has deep experience with automotive visualization and offline rendering, as evidenced by the PSA car configurator and projects with Dassault Aviation. Although we primarily used a different real-time engine until recently, we have now partnered with Unity to deliver real-time rendering results using HDRP that achieve new levels of visual quality and performance. To prove these new capabilities, we recently created a video demonstrating real-time rendering of a Lexus LC500.
The project from start to finish
CAD data preparation tools and process
For the LC500 project, Lexus provided us with access to boundary representation (B-rep) CAD data for all visible surfaces and certain internal geometry. This was helpful, as we didn’t need to do any hard surface modeling. However, the model is very complex and so organizing and preparing the data was a challenge. To meet this challenge, we selected PiXYZ software for its advanced tessellation and scriptable data preparation features.
As is typical of most complex products, the model of the Lexus is organized into an extensive hierarchy of objects, in this case representing thousands of components of the car. We decided to separate the outside and inside of the car, which gave us more flexibility and allowed us to work on the two parts of the model in parallel. We were also able to segregate design content including the powertrain, chassis, and vehicle structural components.
The source data is not organized as a single vehicle per file, but instead as a collection of parts covering all the geometric options of the same vehicle. This provided us the opportunity to organize the data in a logical way to mirror the variants (options) available to a Lexus customer. We used an XML file to logically connect each part in the source data with the relevant option logic. Using this XML together with a custom script, we were able to isolate the different variants of the vehicle into a form where a visual representation of each option combination can be readily assembled.
Once the different parts were isolated, the next step was tessellation. This process involves transforming CAD data (B-rep) into a tessellated form (triangles) which can be used in applications such as 3ds Max or Unity. By using PiXYZ software, we were able to produce relatively lightweight tessellated models and still produce excellent visual quality.
Options to achieve optimal lighting and baking
To achieve the very high standard for visual quality that we set for ourselves, we used lightmaps to enhance lighting. As part of this workflow, we needed to unwrap every single part of the car without any overlaps. We used automatic unwrapping tools together with interactive (manual) unwrapping where needed to optimize seam placement.
We evaluated two options to compute the lightmaps – directly in Unity with its built-in lightmapper, and in 3ds Max using a third party renderer such as Octane or V-Ray. We decided to evaluate both methods so we could compare the quality level resulting from each, and to test the workflow to integrate external lightmaps with Unity’s built-in lightmapper. The interior lightmaps of the vehicle were calculated directly in Unity, and with the right settings, the results were very convincing. The direct lighting is provided by real-time lights, while the indirect lighting is baked. With this technique, we are able to provide realistic visuals even when animating interactive parts of the scene such as turning the steering wheel and opening the doors. Using the built-in lightmapper in Unity was very straightforward and yielded excellent results.
We chose to experiment with the Octane lightmapper to interactively tailor specular occlusion for each exterior object in the model. We edited the standard shader in HDRP to properly incorporate a separated UV channel. The result was satisfying and enabled us to generate a visual result which is very realistic, especially in gaps between the body panels. Using these two approaches allowed us to evaluate tradeoffs between the flexibility of an external lightmapper and the ease of use of the integrated lightmapper. Ultimately we concluded that we could achieve the desired results with either approach.
Tailored materials to achieve photorealism
With our years of experience in the world of automotive offline rendering, we have accumulated a large collection of high-res textures in the form of diffuse maps, height maps, specular maps, normal maps and more. For this Lexus project we wanted to use our favorite maps to get the best results, but to optimize performance Unity needs metallic, smoothness and AO maps combined into the RGB channels of a single map. This approach is optimal for performance but creates challenges to interactively tweak maps and shaders precisely. Because the Shader Graph wasn’t available for HDRP when we started this project, we asked our development team to create a custom texture editor tool allowing us to load and tweak each map in our textures independently and directly in the standard HDRP shader. This editor tool enabled us to work very efficiently by fine-tuning materials directly in Unity. Now that the Shader Graph is available for HDRP, this same capability is available to all Unity customers.
Post-processing is essential to give a film-like and realistic feeling to a 3D scene. HDRP provides many available options to tweak the final look of a scene, including color grading, bloom, vignetting, depth of field, and more.
Depth of field is an essential visual effect to achieve realistic visualization. It helps the viewer to focus on a specific area and feel immersed in the scene. Applying fixed values for the depth of field focus point within a real-time app could produce unnatural results, so we made a small camera script that automatically sets the focus on the closest object in front of the camera. It was very easy in Unity for us to cast rays within the scene to determine the closest object.
We used Cinemachine in Unity to make what we call a “Demo” mode, a kind of cinematic showcase mode with predefined camera paths, to show key features of the product even when nobody is actively interacting with the car.
To accurately represent the car, we provided a way to manage its many different configurations. Not only did we need to switch among variants in the runtime (executable) app, but also within the Unity Editor, to ensure that all variants were correctly defined. To achieve this, we created a configuration management script that records “scene states” and provides automatic material assignment.
Our experience with this project has given us the confidence to use HDRP to pursue new customer projects demanding the highest level of visual quality. We are able to deploy new projects very efficiently through the use of PiXYZ for data preparation together with the latest rendering, cinematic, and post-processing tools in Unity and our own in-house custom tools and scripts.
Real-time 3D rendering is transforming the way we develop, manufacture and market products. See how Light & Shadows partners with Unity to deliver real-time 3D results in an on-demand webinar. We will go in-depth the steps that we described in the above blog.
We at Unity would like to thank Light & Shadows for this blog post and for the great work they did on this video. More information about Unity’s solutions for Automotive and Transportation can be found here.