We want to try one more thing… On Friday work on something cool.

I think there are a lot of low hanging fruits in Unity where somebody with drive can just do something cool that pushes us forward. Things that are hard to put in words, but just make sense when you see it done.

So every Friday, developers can work on something cool, something they have been craving to do for a long time. A feature, a demo, a tutorial, a video tutorial, docs, cleaning the toilet, something on the website. Whatever. Be creative, push the envelope.

In other words, we started to work on “anything cool” on Fridays. This was dubbed as FAFF (Fridays Are For Fun).

Boy, what a positive effect on team morale that was!

It also resulted in some FAFF work being polished & promoted into official releases already. Things like:

• Animation Events in Unity 2.6 animation editor
• Display realtime brush preview in terrain editing tools
• Timeline Charts for Unity 2.6 profiler
• Custom Icons for standalone game builds
• Head Look Controller sample project
• Precise raycast-based snapping in Unity 2.6

…all started out as Friday projects!

Developers have experimented with lots of other stuff on Fridays. Some of them were just playing around or learning Unity better. Some Fridays were for evaluating various middleware packages, implementing some features or doing some fancy R&D. Some of the projects were an interesting kind of “fun”. Like improving internal build farm setup, a system for better tracking of crash reports, improving the website, or other similar things that often involve Perl scripts. Well, eveyone has his own kind of fun

So yeah. On Fridays, we just ignore the schedule and work on something cool. Something that pushes our own tech – something that we can’t defend from a business standpoint but that’s just too great to not get done. If it rocks, we’ll ship it. Because, at the end of the day – who knows where the next killer idea will come from.

The objective of this Unity Summer of Code project was to develop a toolset for the Unity Editor which would streamline and improve the workflow involved in creating terrains for games.

While there are currently a number of third party tools that can already produce high quality terrains, I felt that they lacked two vital qualities. Besides the impact of the additional cost of these tools – particularly to indie developers, when using these tools it is left to the artist to develop an effective workflow for getting height and texture maps out of their proprietary software and into Unity. This can be a somewhat painful process. Secondly and more importantly, it is impossible to leverage the power of the tools to procedurally generate landscapes at runtime. My aim was to overcome these constraints by integrating terrain generation tools directly into the Unity engine and editor.

The toolset I am developing is divided into three areas – terrain generation, erosion and texturing. So far there are 5 different erosion filters – thermal, tidal, wind and two types of hydraulic erosion, 2 ‘generators’ that use Voronoi and cloud fractal algorithms to create base terrain, a couple of useful tools that allow the user to smooth or normalise the terrain and finally a procedural terrain texturing tool.

All ten of these features will be provided both as filters which can be applied in the Unity Editor and as fully documented API functions which will allow users to generate terrain and apply filters at runtime through their own scripts. Additionally, thermal, fast-hydraulic and tidal erosion filters can also be applied directly to the terrain object using brushes in the editor.

Where it’s up to, what still needs to be done

I feel that the project has progressed well over the last five or so weeks. There have certainly been some obstacles and not everything has gone my way, but with just over a week remaining I am on track towards delivering at least 16 of the 19 features that I originally scoped for the project, plus two completely new ones. Given that my aim was to deliver a minimum of 12 of the scoped features within the Unity Summer of Code timeline, I’m quite happy with the result.

Over the next week my main aim is to tidy up the documentation for this project, which I believe is vital in allowing Unity artists and developers to get the most out of these tools. I will also focus on cleaning up my code and optimising it where possible and finally I will continue working away on the remaining features – namely a few UI tweaks, yet a third different approach to hydraulic erosion, and glacial erosion.

What went well

I started the project with a fairly limited knowledge about creating properly integrated custom tools for the Unity Editor and had only given the Editor GUI documentation a cursory glance. I found that despite a few minor hurdles, I was able to quite easily create a useful and intuitive interface for my tools. I was also pretty happy with some of the optimisations I was able made to my erosion code. Being able to run 50 thermal erosion iterations on a 512 x 512 terrain object in barely more than 6 seconds totally exceeded my wildest expectations.

Also, after many unsuccessful attempts to get Unity’s undo feature to work with my code, one of the Unity guys, Rune, was kind enough to explain it to us – and in typical Unity style, it turned out to be much simpler that I had thought it would be.

Even though I had originally considered undo as a ‘nice to have’ feature, I quickly realised that it was really an imperative. Getting the best results from these tools can require a bit of trial and error, and allowing users to run a filter, undo it, tweak a few settings and run it again without completely ruining their terrain object in the process is vital in making this project a practical solution for Unity artists and developers.

What didn’t go so well

While I consider my scripting skills in JavaScript and C# to be reasonably good, I have to admit that mathematics is not really my strong point. Some of the equations I used in my tools, such as Voronoi diagrams and the Diamond-Square algorithm were fairly easy to implement in code – but for the vast majority of my tools I initially approached the algorithms logically rather than algorithmically.

For the most part this worked well and produced good results. However, when I started looking for a more physically correct approach to erosion caused by fluid mediums – namely water and ice – I realised that the complexity of the mathematics involved in computational fluid dynamics, such as Shallow Water Models, Navier-Stokes equations and so on, was far beyond my level of understanding and well beyond my capacity to learn in such a short period. This was rather disappointing, as it has meant that I have not yet found an effective solution for glacial erosion.

That said, there is still a lot more that I want to achieve with this toolset. After I deliver my project at the end of the Unity Summer of Code, I fully intend to revisit some of these prohibitively complex features once I have the time to do all the learning and research that will be required to get them to work the way that I want them to.

I talked about developing graphics technology for small games (PDF slides). Mostly on hardware statistics, GPU features, testing and stability.

ReJ talked about low level iPhone (pre-3GS) rendering details (PDF slides). On inner workings of iPhone’s GPU, OpenGL ES drivers, command buffers, VFP assembly and so on. Some awesome tech stuff in there!

If you’re going to watch some demos from Assembly 2009, make sure to see:

• Frameranger (1st place demo). Imagine that on a big screen and a good sound system!
• The Golden Path (3rd place demo) – for something fresh.
• Muon Baryon (1st place 4 kilobyte intro) – that’s what kids do with sphere marching on the GPU these days. Everything in 4 kilobytes!

…see you at Breakpoint next year?

Currently Unity mixes the fixed function pipeline for vertex lit objects and vertex shaders for pixel lit lights. On Direct3D this creates some rendering artifacts on very close or self intersecting surfaces. We want to end this by implementing a full fixed function emulator in vertex shaders. You won’t have to do anything, it will just work!

Well, the good news is that we have done it, and it is definitely coming in Unity 2.6!

It is funny, because the problem seems like a simple one: “there are double-lighting artifacts on close surfaces”. The solution, however, is “we need to implement whole fixed function pipeline manually”, which involves combining shader assembly fragments, doing register allocations, packing vertex pipeline state into bits and other low level hacking.

In other words, it required quite a bit of thinking and implementation effort, so I decided to write up a technical report on how we solved it. If you’re a graphics programmer or love programmer speak or just have nothing better to do: here’s the report.

Unity 2.5 was the first version of the editor running on Windows and eventually was released in March 2009. Some development of it started at end of 2007, just after Unity 2.0 was released. First there was some serious code shuffling, converting from Objective C into C++, refactoring into platform independent interfaces and similar highly invisible stuff.

Here’s the first Unity running on Windows screenshot I could find (this is February 2008):

This was the version dubbed “Can Haz Menus”. However, a couple of days later it was “CanHazGUI” already:

That’s true, whole editor at that time only had a view into the Project, and you could not really do anything with it, except to use the scrollbar. Or maybe collapse and expand the folders – I forget. Things were progressing really fast though. About a week later I sent this status report mail:

Working on Windows editor. Not much to say, so I’ll attach a cute picture instead. It’s getting there.

(Back then I was the only person with a working 2.5 for Windows build, and I did work separately from the main office. Occasionally I had to send screenshots as an argument for getting my salary!)

…and then came work for Unity 2.1, Unity iPhone, Fusion Fall and whatnot. Work on 2.5 was progressing, but at a bit slower pace. And of course, “getting something on screen” is much less work than “getting it to actually work”.

Here’s how the Windows editor looked in September 2008:

This was approaching final look of 2.5 already. Of course, occasionally we’d run into funny situations where it did not quite work. For example, at that time if you dropped a refractive glass shader into scene, it would turn the editor into this:

It took another half a year to add all new & missing features, fix a ton of large & small issues until it got to a real release:

Well, here they are: unity3d.com/webplayer/hwstats

Operating system versions, desktop resolutions, graphics driver versions, shader models, memory sizes and some more. All broken down by quarter so some sort of “trends” can be seen (sure, changes can be caused by general hardware change or simply different people groups playing different games).

Enjoy!

However, the “casual gamer”, which is what Unity games are mostly targeted at, probably has slightly different hardware. “Slightly” being a very relative term of course.

Lo and behold – we have a glimpse into that data.

How? First time the Unity Web Player is installed, it submits anonymous hardware details (details in the EULA). This happens only once, and contains no personally identifiable information. It’s much like visitor statistics trackers on the websites that gather your OS, browser information and whatnot.

We still haven’t prepared nicely laid out, colored and formatted reports into that data that we can share publicly like Valve, but we’re working on that. So in the meantime I’ll share some images without fancy graphs or colors, bear with me.

Remember, all this data is from people who installed Unity Web Player (most likely because they wanted to play some Unity content on the web). Hardware of standalone game players might be different, and hardware of your game’s players might be different as well. The data set is well over a million samples at the moment.

Enough talk, let’s show some pictures.

What operating systems do we have?

This one should be easy to understand.

What Windows versions are out there?

We should know OS X versions as well, right? Hey, Leopard already took over Tiger!

Hmm, looks like 64 bit Windows haven’t really taken off yet…

Enough about OS, what about CPU?

Poor Transmeta… (for the record, that’s 43 Transmeta CPUs)

Multicore CPUs are taking off (well, at least dual core ones).

Graphics card?

Capabilities of the graphics cards.

The capabilities are not as bad (certainly better than I expected). I mean, about 70% of them support shader model 2.0 or higher!

What is is troubling is the 3.7% that run in OpenGL 1.1 software mode… That means they don’t have graphics drivers installed or have disabled hardware acceleration. This is just crazy, I mean, without the drivers, even dragging windows around is horribly slow… how people even use the computer this way?

Another interesting point is that DirectX 8.0 level hardware (GeForce 3/4Ti) is pretty much dead. I guess that’s because there never was an entry-level graphics card from that range (before there was GeForce 4MX, which is DX7 level, and it was followed by GeForce FX 5200, which is DX9 level).

However, the above was technical capabilities of the graphics cards. Let’s take a look at which cards are out there:

Uh-oh. Can you say “low end”? The first decent card here is GeForce 8600, 15th on the list. Everything above – slow, slow, slow. Some horribly slow. Well, that’s casual gamer…

Here’s a rough performance indicator, graphics card fillrate in gigapixels/second:

For the record, fillrate of GeForce 8600 is 4.3 GP/s, Radeon X1600 is 2 GP/s, Radeon HD 2600 is 2.8 GP/s, and Intel 945 (GMA 950) is 1.6 GP/s. The difference from high-end to low-end in video memory bandwidth is even larger.

On a somewhat unrelated note, here’s DirectX 10 card distribution:

DirectX 10 can only be used starting with Windows Vista, and on DX10 capable graphics card. So that’s 2.6% of the Unity web game players – still not a very significant amount.

In conclusion

So here was the glimpse. We’ll prepare more and nicer reports into the hardware data sometime soon. Stay tuned!

Edit: it’s live – unity3d.com/webplayer/hwstats

Here’s a glimpse into what is needed to get it working.

The Plan, straight from our internal planning wiki, with some explanations added:

o) Convert all terrain code to use Intermediate Mode Render Calls. This makes all the points below possible, and makes things like Projectors or wireframe terrain rendering in scene view pretty much just work.

o) Update terrain shaders for proper rendering order and soft vegetation. Expand ShaderLab so that a shader pass can indicate it should only be executed when Soft Vegetation is on.

o) Proper vertex lights on terrain, implemented in a vertex shader. Currently terrain handles up to four directional lights; we need to expand it to handle point lights as well. I think we’ll treat spot lights (when in vertex lit mode) as point lights; as calculating multiple spot lights in a vertex shader is next to impossible because of instruction count limits. Spotlights are not very useful in vertex lighting mode, right?

o) Implemented pixel lit shaders for terrain. This will get light cookies, per-pixel lit terrain, and ability for terrain to receive shadows.

o) Add option somewhere for pixel or vertex lit terrain (defaults to vertex lit).

o) Proper vertex lights on trees. Very similar to “vertex lights on terrain” point above.

o) Make tree shaders able to cast shadows. Because trees are scaled & animated in a vertex shader, the shadow-casting shader needs to apply the same animation as well.

o) Make Projectors not affect grass / trees. How? We’ll expand ShaderLab so a any shader can say “no Projectors on me, thanks”, and modify grass & tree shaders to do that.

o) Make projectors not affect terrain when game content is built with Unity 2.0. This is web player backwards compatibility – there are Unity games out there that use projectors and terrains, and they should continue functioning exactly like they did before. So Projectors should not affect terrain in this case.

o) Make tree shadows not s*cklook bad when they are squashed for billboarding. Trees are squashed into a planar shape when they approach billboards; this is to make the mesh-to-billboard transition less noticeable. However, this makes shadows from those trees look very wrong. Not sure how to handle this; I think we’ll just stop casting shadows when trees are starting to squash.

o) Redo Lightmapped terrain shaders:

• Use existing shaders for game content built with Unity 2.0. Again, existing web player games should work like they did before! For reference, in Unity 2.0.2 lightmapped terrain was not affected by any lights at all.
• Make new set of terrain shaders that follows the convention of other Lightmapped shaders: lightmap, plus additional lights on top. Whoa, hey, we need two new sets of those shaders: for vertex-lit terrain and for pixel-lit terrain!

o) Optimize the bejesus out of rendering internals. Why – because the old terrain rendering code was like this:

1. Figure out which parts of terrain and vegetation are visible.
2. Set terrain shader.
3. For each terrain chunk: draw it’s mesh!
4. Set tree shader.
5. For each tree: draw it’s mesh!
6. …similar for billboarded trees, grass, and so on.

This is all simple & nice. Of course this was exactly the problem why shadows, projectors and such didn’t work on terrain – everything was just rendered immediately. Now, the new terrain code just submits any visible parts of terrain as Intermediate Renderers, and the rendering code figures the rest out… But that means it’s going through generic rendering “pipeline”, that has to figure out lighting, sort out objects, set shaders and their parameters, etc. The major overhead in switching to this “proper rendering” is the lost ability to do this:

1. Set the shader.
2. Draw lots and lots of objects with it.

The rendering code in Unity does sort objects by shader, and does ultimately figure out that they are using same shader, possibly same set of textures, even possibly same set of lights and same set of material parameters – but some time is spent just figuring out that nothing actually changed between the objects.

Well, the upside is that optimizing rendering internals makes everything else run somewhat faster as well!

(I am here right now… the points above are done, the points below are on the list)

o) Make hills/trees outside of view be able to cast shadows (how?)

o) Expand shaders of detail meshes so they can cast shadows.

o) Optimize terrain & tree vertex lit shaders. Right now they process four vertex lights, where each can be directional or point light. This is noticeably slower than old shaders, that only supported directional lights (point lights need more calculations). We need to write multiple combinations of those shaders, specialized for specific situations (”one directional light”, “two direcitonal lights”, “two arbitrary lights”, and so on).

o) Add option somewhere for terrain to not cast shadows.

o) Add option somewhere for trees to not cast shadows.

o) Add option somewhere for detail objects to not cast shadows.

o) (maybe, don’t know how to do) Option for grass to receive shadows.

o) Reduce memory allocations in terrain code. Because of intermediate rendering, some of temporary arrays are not actually needed anymore.

Whew, that was quite a list! I’m not sure I’m able to even read through all of that… well, back to work.

…
Joachim: yeah it’s awesome
Joachim: function Start () { /* super secret code snippet */ }
Aras: whoa
Joachim: it rocks
Joachim: automatically instantiates material for you
Joachim: the same way we do it for scripts
Joachim: best thing since sliced bread i think
Aras:
Aras: hey, back in the day the terrain was best thing since sliced bread!
Joachim: sliced bread appears to be always worse depending on what i work on.
Aras: maybe sliced bread is just not so good in fact
Aras: sliced bread is overrated

And no, I won’t say what we were talking about

Wait a moment… Unity plugin makes nice wobbling web page elements not wobble anymore!? Sounds like an interesting issue…

So I prepared for a debug session and tried the usual “divide by two until you locate the problem” approach.

• Unity Web Player is composed of two parts: a small browser plugin, and the actual “engine” (let’s call it “runtime”). First I change the plugin so that it only loads the data, but never loads or starts the runtime. Everything works. So the problem is not in the plugin. Good.
• Load the runtime and do basic initialization (create child window, load Mono, …), but never actually start playing the content – everything works.
• Load the runtime and fully initialize everything, but never actually start playing the content – the bug appears! By now I know that the problem is somewhere in the initialization.

Initialization reads some settings from the data file, creates some “manager objects” for the runtime, initializes graphics device, loads first game “level” and then the game can play.

What of the above could cause something inside browser’s JavaScript engine stop working? And do that only on Windows, and only on some browsers? My first guess was the most platform-specific part: intialization of the graphics device, which on Windows usually happens to be Direct3D.

So I continued:

• Try using OpenGL instead of Direct3D – everything works. By now it’s confirmed that initializing Direct3D causes something else in the browser not work.
• “A-ha!” moment: tell Direct3D to not change floating point precision (via a create flag). Voilà, everything works!

I don’t know how I actually came up with the idea of testing floating point precision flag. Maybe I remembered some related problems we had a while ago, where Direct3D would cause timing calculations be “off”, if the user’s machine was not rebooted for a couple of weeks or more. That time around we properly changed our timing code to use 64 bit integers, but left Direct3D precision setting intact.

Side note: Intel x86 floating point unit (FPU) can operate in various precision modes, usually 32, 64 or 80 bit. By default Direct3D 9 sets FPU precision to 32 bit (i.e. single precision). Telling D3D to not change FPU settings could lower performance somewhat, but in my tests it did not have any noticeable impact.

So there it was. A debugging session, one line of change in the code, and fancy javascript webpage animations work on Windows in Firefox and Safari. This is coming out in Unity 2.0.2 update soon.

The moral? Something in one place can affect seemingly completely unrelated things in another place!