Categories & Tags
Archive

Advanced Shading and Lighting for Mobile

June 8, 2011 in Demos, Tutorials and Tips, Technology by

 

Hello everyone. You have probably already heard about Shadowgun, the new mobile game developed by MADFINGER Games. You might also have seen Shadowgun being demoed at WWDC on stage during the Graphics State of the Union talk.

Needless to say, we’re very proud of the next great creation made in Unity, and we love how it delivers console quality to the mobile with the help of Unity’s awesome tech. It runs at an impressive 60fps on the iPad 2 and 30fps on the iPad, iPhone 4 & 3GS. It will also be released on Android. Since lighting is one of the main elements making the game look so good, here are some notes about the techniques we used.

Character shading

All characters in the game use the same per pixel normal mapped isotropic BRDF pixel shader, which goes beyond the usual Phong shading. It gives a great effect on the skin of the character, metal and cloth. Our solution combines off-line precomputed texture which describes how surface interacts with lighting and run-time per-pixel calculations to achieve both artistic control and superb performance.

Light probes

shadowgun-greenLight probes are an advanced technique for lighting dynamic objects and characters with high-quality baked lighting. It is possible to capture incoming direct and bounced lighting within a scene at a number of points – light probes. As a character moves through the scene, nearby probes are picked, interpolated, and the result is then passed to the shader to efficiently light the surface. There it can be sampled per-vertex or per-pixel, if you want to use the normalmap-perturbed normal to do so.

Choosing and interpolating the right probes (and doing it fast!) is a hard problem, though. We have developed a technique, which takes an arbitrary set of probes and performs Delaunay tetrahedralization on them to divide the entire space into tetrahedron-shaped cells.

Shadowgun_muzzleThat gives us fast interpolation with great properties and no light popping – when you’re in a given tetrahedron, weights for the four corner probes are your barycentric coordinates! Finding the right tetrahedron can also be done efficiently by checking the one from last frame – if it’s not a hit (most of the time it is!), we check it’s neighbours. Finally we make sure interpolation works outside of the volume of the probes just as fast and as good with a clever projection onto the hull surface.

I’m especially proud of the light probes, since we managed to shape them up to be extremely fast and have low memory footprint – perfect even on mobiles and giving so much bang for the buck. As far as we know, it’s also the first time it has been used to such extent (at all?) on mobile devices.

Muzzle flashes

As a bonus, since we’re already rendering every character with a light probe, we can project any additional lights onto the probe as well. We do that for the muzzle flash effect. For a very low CPU cost and no GPU cost characters get lit by dynamic light.

God rays, light shafts, light flares and fog

God rays, light shafts, light flares and fog are rendered using both static and animated alpha-blended geometry with very optimized vertex and pixel shaders to save fillrate.

Lightmaps

Lightmaps were baked with one click in Unity using Beast to capture all the beautiful Global Illumination on static geometry. To give the objects a more detailed look while still keeping the shaders extremely fast, we calculate approximate specular reflections per vertex and then modulate them per pixel with a gloss mask to differentiate between shiny and rough parts. Reflective objects additionally use a pre-baked cube map matching the surroundings.

Caustics and character shadows

Caustics from the light passing through the green fluid, character shadows and rotating ventilator shadows were simulated with textures projected onto tight geometry (to keep the fillrate requirements low) and then animated in the shaders.

shadowgun-in-game-unity

We plan to release light probes with Unity 3.5; all the other effects can already be achieved in Unity! :)

Share this post

Comments (54)

Comments are closed.

8 Jun 2011, 12:58 am

:O Very nice! 3.5 though! 3.4 isn’t out yet and your already tormenting us with things we will have to wait longer for :)

JinWanDaLaoHu
8 Jun 2011, 1:21 am

What’s the BRDF pixel shader,is it in the Unity default shader?

8 Jun 2011, 1:36 am

That is so impressive.
I see you solved the tetrahedra interpolation quite nicely :)

And I just saw your answer in the old thread on probes. Great to hear you are exposing enough for probe painting and implementing probe prefab.
On 3. “setting intensity of probe to a minus value to create shadow” I didn’t specify I wasn’t talking about cast shadows but shade, as in the darkness within a cave that comes from occlusion from the sky and sun lightsources. We used a trick back in the days of softimage 3.6, before we had ambient occlusion, where we’d place a negative light inside a cave to darken everything in it. Maybe it’s not possible for the same reason we can’t have negative lights – compatibility with the old ATI.

Anyway thanks for creating the tools for cheap artistic lighting !!!

TJHeuvel
8 Jun 2011, 2:12 am

So all we need to do is write a per pixel normal mapped isotropic bidirectional reflectance distribution function pixel shader? Sounds easy!

8 Jun 2011, 2:22 am

@TJHeuvel: yes, piece of cake! Actually we plan to put the editor scripts to generate lookup textures & the shader itself onto Asset Store for free soon. Just didn’t get around to actually do it yet (everyone trying to ship Unity 3.4, or something…)

8 Jun 2011, 2:58 am

Very nice work! love to see some rocking graphics inside unity!

rean
8 Jun 2011, 3:13 am

3.5….!!!???? ow come on guys T_T……. how about 3.4 :p

thienhaflash
8 Jun 2011, 5:23 am

Very impressive ! I really like the light probe idea ! Unity3D is shaping 3D’s future, yay !

Toha Yamaji
8 Jun 2011, 5:31 am

Now we have a link to paste every new forum topic about “Unity vs UDK: graphics”.

Dev.D
8 Jun 2011, 5:58 am

A couple of question:
1. does the BRDF simulation work equally well on the deferred rendering path as well?

2. An specific requirements for the LUTs texture formats? I’m hope since this is actually working (great) on the mobile – you guys aren’t using anything exotic.

Overall excellent news :) . I’m glad we are getting a tweak-able BRDF system (shader & perhaps scripts) embedded into Unity. And am glad you are doing it for us . This is great news indeed as a single BRDF material can encapsulate multiple physical materials :). Our graphics guys used a similar system on APB for our customized character reducing the need for multiple materials per character to one . Great for things like fur , leather , skin esp when they appear on the same character … exciting news!! ^__^.

-Dev

8 Jun 2011, 6:47 am

Wow!,

Awesome job Robert & Kuba! :D

Jason Amstrad
8 Jun 2011, 7:09 am

so what will be new for unity 3.4 then ?

JoeW
8 Jun 2011, 8:08 am

Ah – suck. Lightprobes have to wait until 3.5? Oh well, all good things come to those who wait, eh? I’ve still got a ways to go before I’m ready to implement them anyway…. sigh.

Nice work, guys – I love the fact that Unity is being so aggressively developed, and I certainly appreciate all the hard work.

Jason Amstrad
8 Jun 2011, 8:22 am

Unity was working on an internal pathfinding solution if I’m not mistaken.
How about some news on that ?

Manon Seppen
8 Jun 2011, 8:24 am

It would be nice if unity also supported material instancing.
And also dynamic fluid surfaces and wave generation,if a character moves through the surface for example.

Dirk41
8 Jun 2011, 8:26 am

@Jason Amstrad

what does material instancing mean ?

Jon
8 Jun 2011, 8:35 am

“God rays, light shafts, light flares and fog are rendered using … with very optimized vertex and pixel shaders to save fillrate.”

Any chance we can get a look at that shader? I am really curious how it solves/helps the fillrate seeing as how the retina and iPad are usually fillrate limited.

8 Jun 2011, 11:45 am

@Dev.D:
1) the exact shader used in this game would be rendered in forward path. Most of interesting lighting calculation in deferred is done by a single shader (thus it’s hard to make really different materials there), but you can surely override the shader that’s used in Unity by default. For example, replace the current Blinn-Phong with a properly energy preserving one, etc.
2) Well, a much less fancy sounding name for that texture would be “a lighting lookup texture indexed by (N.L, N.H)”. Turns out doing an extra texel fetch is faster on current mobiles than computing pow() + couple more math instructions. And it allows more flexibility than standard Blinn Phong BRDF. It’s still limited to isotropic ones, and the ones that can be indexed by (N.L, N.H) of course. The concept is explained in this article.

@Jason Amstrad: you know, I was thinking “hey we do a development related blog post, Jason will come and ask for pathfinding”. I was right! ;) Yes, we know you want pathfinding, and we are working on it. This blog post is very much not about pathfinding though. Can we keep it on topic please?

@Jon: my guess on the “very optimized shaders” – well, shaders that do almost nothing. For example (just guessing), maybe some of those “special fx planes” don’t even use textures. The vertex shader would then only do one matrix multiply to transform the vertex position, and the pixel shader would only return a color. To save additional fillrate you can employ other tricks, for example when camera gets close to some “god ray plane” object (so it would cover large area of the screen) – scale it down and fade it’s color our. This is not “correct” (objects shouldn’t become smaller when getting closer to them), but saves a ton of fillrate!

Julio Sanchez
8 Jun 2011, 2:27 pm

What do you mean by “animated in the shaders”? when you talk about the rotating ventilator shadows.

Best Regards

ZroeX
8 Jun 2011, 3:17 pm

Wow!!
How amazing!!
The next update to 3.5 directly?

By the way,the game demo is look like “Gears of War”.
^__^

Hanzo
8 Jun 2011, 3:53 pm

Come on!
That looks like a Gears of War or Bulletstorm clone! Do we really need more bland and unoriginal games with lame art?

Giyomu
8 Jun 2011, 4:10 pm

@Hanzo, what the point of this stupid reflexion ? You are free to make your own original no bland game with no lame art :)

and for UT guys well as usual nice improvement on the engine, the mobile tech become really powerful, well worth wait for 3.5 imho.

Fractal Core
8 Jun 2011, 8:18 pm

Wow, Giyomu doesn’t get it, at all. This is a demonstration of the technology being used by the game. And by your own admission you don’t even WANT to make your own game. What the hell are you doing on a unity blog leaving a REVIEW of a game you haven’t played.

We all know the brown-and-grey-type-of-game-is-unoriginal argument… Boring. Try posting what you think of how well Unity is matching something done in UDK.

And before mentioning Infinity Blade, go find out which techniques they’re using inside UDK, then come back and find out what Unity can do. It comes down to what the artists can do with the technology, and something very close to Infinity Blade could easily be done with Unity. Not that you’d care, since you’re JUST a “PLAYER”

Fractal Core
8 Jun 2011, 8:20 pm

Sorry about that Giyomu, I got the name wrong. I meant Hanzo is the lame-ass “PLAYER”

8 Jun 2011, 9:03 pm

Wow what can I say. I think this looks fantastic, and by far the best looking game on the asset store. at least 50% of the result is great art though. The shaders would be nice for unity pro users, an incentive to upgrade. Fantastic work, certainly better looking than infinity blade but thats a personal observation.

Finally, I’m very tired of children posting comments into blogs or forums.

Ashkan
8 Jun 2011, 10:37 pm

awesome skin shader.
lightprobes are really a great feature. waiting for features is something natural, people who they are working on it are humans like us believe me!!!
they need time to do their jobs, they are not God or super fast robots who could do coding!!! :)

8 Jun 2011, 10:50 pm

Oh, and everyone asking about Unity 3.4 vs 3.5: we’re at the finishing stages of Unity 3.4 release… we aren’t skipping it! Light Probes will come into the release after that (3.5).

Greg
9 Jun 2011, 1:17 am

I am curious, are the light probes modeled using spherical harmonics ?

9 Jun 2011, 1:35 am

@Greg: yes. And since Unity already applies spherical harmonics lighting in forward rendering (all “not important” lights + ambient goes there), taking light probes into account has no additional cost at all on the GPU. There’s only very small cost on the CPU per object (to find, interpolate the probes and add them to SH info).

PedroAfonso
9 Jun 2011, 2:23 am

@Aras, what features will there be in 3.4?

Since UT is so close to release it, could you give us a preview of some features it will bring?

9 Jun 2011, 2:38 am

@PedroAfonso, watch our blog. Soon.

Dirk41
9 Jun 2011, 4:45 am

@Aras Pranckevičius
How soon ?

Baronium
9 Jun 2011, 5:01 am

@Dirk41
Aras said soon. This is soon enough.

I’m excited what opportunities we have with those light probes in the future.

Edinson
9 Jun 2011, 7:07 am

This is awesome, good bye UDK!

PeterB
9 Jun 2011, 7:35 am

Hm. There’s at least one package in the Unity Store which requires Unity3d v3.4.

Curious
9 Jun 2011, 11:07 am

Hmm, this is great :D
Can’t wait for 3.5 though ;)

We’re waiting for the preview of 3.4 guys ;)

Damien
9 Jun 2011, 2:28 pm

Will this light probes feature be just for mobile platforms, or will it included into Unity for desktops and web apps as well?

Please say it will be … pretty please!

PedroAfonso
9 Jun 2011, 2:48 pm

it will probably be!

9 Jun 2011, 3:24 pm

@Damien: Light probes will be available across all platforms of course.

@Baronium, @Laurent: I’m excited to see what tools the community will come up with to allow artists to paint lighting into the probes or other funky stuff like that :)

Holland
9 Jun 2011, 6:40 pm

Did I just saw penis on the statue? How are they going to get pass the Apple censors?

9 Jun 2011, 9:45 pm

@Manon Seppen: we don’t know, so the best answer would be “maybe someday”. Also, if you have “instanced draw calls” in mind, you’d have to know they don’t work (or don’t bring any performance benefits) on all platforms. On PC with Direct3D – sure. On consoles? Not so much (draw calls are really cheap there). On mobiles? They don’t exist.

pixnlove
10 Jun 2011, 4:55 am

At the moment I am unable to use NormalMap with LighMapping.
Normal Map just do not show when Lightmap are unable.
Does that mean we will be able to do that soon?

You Unity3D guy are too good to be true, carry one the Great work.
You madfinger guys are too talented I wish I would be able to design
a game that look like, that is pure awesoness!

Nathali Abbortini
10 Jun 2011, 5:24 am

Are “light probes” the same thing as what the unreal engine calls “light environments” ?

10 Jun 2011, 5:57 am

@Nathali: the basic concept of sampling the irradiance in space and then using the samples to light the dynamic objects is similar. But the execution is completely different.

It uses dense uniform grids within a bunch of bounding boxes placed around the scene. I don’t know how it samples the data, but my best guess is that it finds all the probes within a given range and assigns weights to each one – or discards it if a raycast fails. If that’s the case, it suffers from the resulting interpolated probe being very blurred and failing to show contrasts in the lighting, even though they were correctly captured by the baked probes. That kind of interpolation is slow and a regular grid takes up a ton of memory.

On the other hand our solution gives you probably the best possible interpolation quality, is fast, has a low memory footprint and runs on mobiles with no problem :)

Nathali Abbortini
10 Jun 2011, 9:11 am

@Robert Cupisz
Nice to hear that it runs faster than UDK.
You guys are the best.
Thanks Unity Tech.

Matt
11 Jun 2011, 8:09 am

Very impressive stuff!

It’s going to make the wait for the updates to 3.4 and 3.5 even harder now though!! :)

Jason Amstrad
28 Jun 2011, 6:01 am

@Aras
First no pathfinding until 3.5 and now the promised dynamic fluid surfaces are not on the roadmap at all. I’m sorry but this is simply unacceptable. Cheaper engines like UDK have more features than this

28 Jun 2011, 6:42 am

@Jason: seriously, if you really think that Unity’s features or the roadmap is “simply unacceptable”… why not use something else? We try to work on various features as hard as we can, and to prioritize them as good as we can. If all our work in simply unacceptable for you, then oh well, maybe Unity is just not for you. Good luck!

Aj
29 Jun 2011, 12:08 pm

Unity Smashed it AGAIN! It seems like Torque and Unity are really Striving to Be the BEST!

Rodrigo
29 Jun 2011, 5:50 pm

uOww!

WillBellJr
30 Jun 2011, 1:35 pm

All these changes really sound great!

I’m still hoping for whatever route you take with the GUI improvements, I’ll be able to do this:
http://forum.unity3d.com/threads/23336-Better-2D-Graphics-Support

29 Jul 2011, 10:39 am

I hate to sound ungrateful but I’ve just installed Unity 3.4 and now the OTHER half of my shaders are broken.

It would be good if Unity could provide us with rewritten ones to replace the broken ones as quite a lot of people have no clue as to how to code Shaders. Including me.
Thanks

29 Jul 2011, 11:44 am

OK, have found some of the rewritten shaders, but would instructions kill you? How do I get my boat floating again and my underwater effect back? Never mind the God rays. Thanks though for some splendid stuff.

20 Aug 2011, 4:04 am

gokillllllllllllllllllllllllll

Leave a Reply

Comments are closed.