Photogrammetry in Unity: Making Real-World Objects into Digital Assets
Photogrammetry is the process of using multiple photos of the real-world objects to author game-ready assets. It’s best suited to objects that are time-consuming to produce in 3D sculpting software. This post explains how new Unity features assist you in working with photogrammetry. We’re also sharing our Fontainebleau photogrammetry demo project, including all meshes, textures and materials.
Last year at Siggraph 2017, we revealed a highly detailed field guide for professional artists. It takes you through the entire process, step-by-step, for producing high-quality, reusable, and game-ready digital assets from photos and video.
Part of this workflow, the team released the De-Lighting tool on the Asset Store. It enables artists and developers to remove lighting information from photogrammetry textures, so that the final assets can be used under any lighting condition. We also gave a talk at Siggraph about an overview of the photogrammetry workflow and shared technical details about the De-Lighting tool.
Authoring material with Photogrammetry asset for game development
This year, we created a step-by-step guide to shows how to use a layered shader to achieve the same level of quality as shown in the photogrammetry workflow guide, but optimizes on texture memory budget to cater for your gaming needs.
Here’s an overview of the steps of authoring material with Photogrammetry asset in Unity:
Check out the complete guide:
Layering materials in Unity with Hight-Definition (HD) Render Pipeline
Photogrammetry allows you to get a qualitative result, but requires a very high texture resolution to conserve details, as shown here:
This is unpractical for game authoring due to memory budget, and it doesn’t allow you to add any variation to the object.
Unity 2018.1 beta introduced a preview of Scriptable Render Pipeline. We will release two built-in render pipelines with Unity 2018.1: the Lightweight (LW) Render Pipeline and High-Definition (HD) Render Pipeline. HD Pipeline provides a shader dedicated to photogrammetry material authoring name: LayeredLit.
A layered shader defines the visual with a combination of individual materials.You can see the main material in the picture above is mixed with other materials like stone, ground element and moss. These other materials are tileable. This means they can wrap around objects and you can reuse them on different objects. Using a combination of materials enables you to have a similar visual quality as a high resolution texture (as in a similar texel density on screen), but with low resolution textures, which saves memory:
A layered shader allows you to share textures between assets, and to combine tileable materials to add variation. This makes it easy for you to populate a large world at low cost:
We created the Fontainebleau demo to illustrate the photogrammetry workflow and the use of the LayeredLit shader. This technical demo is authored with game development condition in mind: it’s a representative game level and targets the standard PlayStation 4 platform at 1080p @ 30fps. The level represents a part of the Fontainebleau forest and uses a limited set of meshes and textures that are reused with different variation with the help of the LayeredLit shader. There is a playable first person and third person mode to walk inside the forest. Targeting consoles like XboxOne or PlayStation 4 requires consideration of how to get the most from these platforms.
The demo also supports three different lighting condition to illustrate that correctly authored and de-lighted asset work fine in any lighting condition:
Night lighting with lights off:
Night lighting with lights on:
Finally, we included 3 modes to explore the demo:
- Cinematic mode: select your lighting program, then sit back, relax and enjoy the show,
- First Person & 3 Person Modes: these are very rudimentary exploration modes to let you discover the environment on your own, with bonuses in First Person mode.
Fontainebleau is the name of a forest close to the Unity Paris office. The forest is a good subject to speak about photogrammetry. Natural assets are often complex and hard to reproduce realistically. For our artists, it was important to have the subject close to them to do all the tests needed to analyze the best workflow possible for games.
Download the project
The project is intended to work with 2018.3 and uses the 4.8.0 HDRP package.
You can download the pre-built PC stand alone player here.
The project is available for clone only here on GitHub. That means that downloading a zip file using the green button on Github will not work. You must use a version of git that has LFS in order to get the project. Take a look at it and inspect the photogrammetry assets: meshes, textures and materials. You’re also allowed to reuse the assets in your Unity project with no restriction.
At the start, the photogrammetry process can seem to be hard to manage, but the result is worth it.
Hopefully this demo will convince some of you to use photogrammetry to improve the quality of assets and help you speed up your projects.
We’re looking forward to seeing what you make and talking to you about this demo and the photogrammetry workflow in Unity in general in this forum thread.