Search Unity

Photogrammetry in Unity: Making Real-World Objects into Digital Assets

, 3월 12, 2018

Photogrammetry is the process of using multiple photos of the real-world objects to author game-ready assets. It’s best suited to objects that are time-consuming to produce in 3D sculpting software. This post explains how new Unity features assist you in working with photogrammetry. We’re also sharing our Fontainebleau photogrammetry demo project, including all meshes, textures and materials.

Last year at Siggraph 2017, we revealed a highly detailed field guide for professional artists. It takes you through the entire process, step-by-step, for producing high-quality, reusable, and game-ready digital assets from photos and video.

Part of this workflow, the team released the De-Lighting tool on the Asset Store. It enables artists and developers to remove lighting information from photogrammetry textures, so that the final assets can be used under any lighting condition. We also gave a talk at Siggraph about an overview of the photogrammetry workflow and shared technical details about the De-Lighting tool.

Authoring material with Photogrammetry asset for game development

This year, we created a step-by-step guide to shows how to use a layered shader to achieve the same level of quality as shown in the photogrammetry workflow guide, but optimizes on texture memory budget to cater for your gaming needs.

Here’s an overview of the steps of authoring material with Photogrammetry asset in Unity:

Check out the complete guide:
Layering materials in Unity with Hight-Definition (HD) Render Pipeline

Photogrammetry allows you to get a qualitative result, but requires a very high texture resolution to conserve details, as shown here:

This is unpractical for game authoring due to memory budget, and it doesn’t allow you to add any variation to the object.

Unity 2018.1 beta introduced a preview of Scriptable Render Pipeline. We will release two built-in render pipelines with Unity 2018.1: the Lightweight (LW) Render Pipeline and High-Definition (HD) Render Pipeline. HD Pipeline provides a shader dedicated to photogrammetry material authoring name: LayeredLit.

A layered shader defines the visual with a combination of individual materials.You can see the main material in the picture above is mixed with other materials like stone, ground element and moss. These other materials are tileable. This means they can wrap around objects and you can reuse them on different objects. Using a combination of materials enables you to have a similar visual quality as a high resolution texture (as in a similar texel density on screen), but with low resolution textures, which saves memory:

A layered shader allows you to share textures between assets, and to combine tileable materials to add variation. This makes it easy for you to populate a large world at low cost:

Fontainebleau demo

We created the Fontainebleau demo to illustrate the photogrammetry workflow and the use of the LayeredLit shader. This technical demo is authored with game development condition in mind: it’s a representative game level and targets the standard PlayStation 4 platform at 1080p @ 30fps. The level represents a part of the Fontainebleau forest and uses a limited set of meshes and textures that are reused with different variation with the help of the LayeredLit shader. There is a playable first person and third person mode to walk inside the forest. Targeting consoles like XboxOne or PlayStation 4 requires consideration of how to get the most from these platforms.

Screenshot from the standard PlayStation 4 1920×1080.

The demo also supports three different lighting condition to illustrate that correctly authored and de-lighted asset work fine in any lighting condition:

Day lighting:

Sunset lighting:

Night lighting with lights off:

Night lighting with lights on:

Finally, we included 3 modes to explore the demo:

  • Cinematic mode: select your lighting program, then sit back, relax and enjoy the show,
  • First Person & 3 Person Modes: these are very rudimentary exploration modes to let you discover the environment on your own, with bonuses in First Person mode.

Why Fontainebleau?

Fontainebleau is the name of a forest close to the Unity Paris office. The forest is a good subject to speak about photogrammetry. Natural assets are often complex and hard to reproduce realistically. For our artists, it was important to have the subject close to them to do all the tests needed to analyze the best workflow possible for games.

Download the project

The project is intended to work with 2018.3 and uses the 4.8.0 HDRP package.

You can download the pre-built PC stand alone player  here.

The project is available for clone only here on GitHub. That means that downloading a zip file using the green button on Github will not work. You must use a version of git that has LFS in order to get the project. Take a look at it and inspect the photogrammetry assets: meshes, textures and materials. You’re also allowed to reuse the assets in your Unity project with no restriction.

At the start, the photogrammetry process can seem to be hard to manage, but the result is worth it.

Hopefully this demo will convince some of you to use photogrammetry to improve the quality of assets and help you speed up your projects.

We’re looking forward to seeing what you make and talking to you about this demo and the photogrammetry workflow in Unity in general in this forum thread.

22 replies on “Photogrammetry in Unity: Making Real-World Objects into Digital Assets”

Shame, each time I try to run the standalone player, it crashes with “couldn’t switch to requested monitor resolution” error.

Looks very impressive. It would be good to have added Fontainebleau links in photogrammetry page and maybe forum thread, as this blog will sink down in upcoming weeks and nobody will find the project update…

Assets and rendering looks nice. The scene is extremedly detailed. However the TPS character looks like a cartoon model next to it. On top of that the animations and controlling of the character is blazing clunky which makes the experience uncanny. I would suggest to Remove that mode or make it better. If your gonna do something do it right.

Global answer.

Have in mind that the demo target PS4, 1080 @ 30pfs, not PC.

The demo focus on Photogrammetry asset and allow switching time of day to highlight that respecting the workflow allow qualitative asset under different lighting condition. The third person mode is not the focus of this demo, we have not invest time but added this mode to illustrate that we were having game in mind and allow some fun.

Import time or other thing related to Unity editor are not the topic of this blog post that focus on Photogrammetry.

Crash on exit on PC is a know issue and solved in 2018.1b12 not out yet (and not available when preparing the zip).

If the Third Person mode is not the focus of this demo then just ditch it or improve it. We don’t need to see a character walking around to know it can be used for games. This is Unity, it runs anyware blazing fast.

Guys, huge load times are pretty standard for projects like this one. UE4’s photogrammetry forest demo took something like 8 hours before you could open the editor and move around in the scene

Didn’t you get the memo. You can only complain about Unity in here. It doesn’t matter that other game engines have the same issues.

The Demo are really nice, great quality and performance though.
but yeah it bit blurry in my opinion didn’t TAA have sharpness settings?
Also one thing, at least please make the third person mode nicer. it glitchy as hell guys, easily stuck for no reason, jumpy animation and what’s with the idle animation?
yeah sorry the third person asset might be not really relevant with the purpose of the demo, it’s just bother me a lot. The third person part.

Nice work, but I find the post processing a bit questionable. It seems awfully blurry, almost like a PS3 game. It would be a much better showcase if the visuals were ultra crisp

You can open the demo and in the post process layer, you can change the saturation blending from 0.95 to 0.5. Then compile. If you go under 0.5 you lose a natural video feeling revelling more pixels.

Amplify team is developing the “Amplify texture” virtual texturing plugin for Unity. Another plugin Granite already does that. I would be quite happy to hear from the Unity devs if you guys ever thought of moving into that direction?

Hey Sebastien, is there a way to get it running using steam VR? or even XR? I know performance won’t be good but just wondering how you would approach it?

Comments are closed.