Benchmarking Unity performance in WebGL

October 7, 2014 in Technology

When we port Unity to new platforms, it is always an important question to find out how well it performs on that platform – and to see what we can do to make it perform as fast as possible.

One exciting new platform we are currently working to support is WebGL. WebGL is unique when it comes to performance: all code needs to be cross-compiled to JavaScript; some common performance-enhancing techniques, like multi-threading and SIMD, are not available yet; and we are relying on a completely new scripting runtime, IL2Cpp, to run user script code. So we need to find out: Will it be fast enough to play games? How will different browsers and hardware compare? What build settings will produce the fastest results?

These are the questions we have asked ourselves, and we have frequently been asked by our users as well as by suppliers of WebGL implementations. The most obvious approach is to run existing Unity content in WebGL and measure the frame rates. We found that, for most of the content we’ve tried, frame rates are at least 50% of native builds (depending on many factors, such as the browser being used). You can try two demos of games exported to WebGL here. However, we would like to get precise and reliable numbers. Which areas are particularly slow in WebGL? Which are fast? How do different implementations compare, exactly?

To find out, we have created Unity Benchmarks, a suite of benchmark tests written in Unity. The benchmarks stress different areas of the engine and produce easily-comparable numbers on performance. Click here to try Unity Benchmarks in WebGL in your browser! (Note that some WebGL implementations on windows may get incorrectly high results for the Mandelbrot GPU benchmark, as they won’t render the shader correctly. We are investigating the issue and looking for a fix.)

screenshot

The Physics Benchmark test

Unity Benchmarks has different tests stressing 3D physics, 2D physics, particles, navigation, animation & skinning, object instantiation, scripting, draw calls and GPU pixel throughput. Each benchmark will count how many iterations of a given task Unity can perform in a fixed amount of time. For the reported numbers, higher is always better.

If you run all of the benchmarks, it will also show an Overall Score value. This score is calculated as a weighted sum of all the individual test results. Since the different benchmarks have very different result scales, they each carry different weights in the overall score.  The weights are calibrated to produce similar result scales on my development machine running in WebGL. But that is somewhat arbitrary, so, when comparing results, the individual scores are much more meaningful than the Overall Score!

So, after all the explanations, here are some results (all benchmarks are run on a 15” Retina MacBook Pro 2.6 GHz i7 running OS X 10.10):

chart2.png

chart.png

The scores have been scaled so that Firefox = 1.0, to fit onto a single chart. Higher is better.

Some observations:

  • In almost all benchmarks, Firefox with asm.js is faster than both Chrome and Safari in almost all benchmarks, and is currently the best browser to run Unity WebGL content. We hope to see asm.js support showing up in other browsers in the future as well, however – see here for some interesting comments on this subject.

  • When you are mostly GPU-bound, you can expect WebGL to perform very similar to native code.

  • In some areas, WebGL will actually outperform native code significantly. This is the case for tests which rely a lot on script performance (Mandelbrot and CryptoHash, which both implement their algorithms in C#), as IL2Cpp can produce more optimized code (expect to read more about this in another blog post soon).

  • Native code can still be several times faster than WebGL for areas heavily optimized to use multi-threading and/or SIMD, such as the 3D physics tests (PhysX 3.3 in Unity 5.0 is now fully multi-threaded). Compared to that, 2D physics is very close to parity when comparing Firefox to Native (Box2D is not multi-threaded). We hope that the future will bring SIMD and multi-threading extensions to JavaScript, at which point this may change.

The most important takeaway is, while there are still areas where WebGL is significantly slower than native code, overall you can get expect very decent performance already, and this can only get better in the future. I hope that these benchmarks may help anyone looking to optimize performance of their JS engines and WebGL implementations to run Unity content as fast as possible.

Comments (51)

Subscribe to comments
  1. Mitch Labrador

    December 3, 2014 at 8:51 pm / 

    My overall score on Chrome was around 100,000 and about 32,000 on Firefox. Chrome peaked at around 185mb memory utilization, Firefox at around 450mb.

    I figured you might find this interesting.

  2. user

    November 25, 2014 at 9:11 am / 

    http://screencast.com/t/s0sAveKy42u
    out of memory
    Ram – 8Gb
    Cpu – i5 2500k

  3. Clowerweb

    November 22, 2014 at 9:12 pm / 

    “In almost all benchmarks, Firefox with asm.js is faster than both Chrome and Safari in almost all benchmarks, and is currently the best browser to run Unity WebGL content.”

    Based on my findings (below) and past experiences, I wholeheartedly disagree with this sentiment. It’s not even subjective – it’s 100% objective. The best platform for a program is not always the one that performs a little better, it’s the one that works consistently with as few breaking bugs as possible. With Firefox, there has never been consistency; something that works today might not work in the next release. Maintaining consistency has always been something that Mozilla has struggled with for some reason. Therefore, Chrome is the best browser to run WebGL applications, because if you use a feature today, you can count on it still working tomorrow. The best platform is always the one that works, and the one you can expect to continue working properly.

  4. Clowerweb

    November 22, 2014 at 8:49 pm / 

    Firefox 33.1.1 wouldn’t run Mandelbrot (just showed a green square with ever-increasing bottom right number infinitely). Chrome 39 completed the test with ~58k on AMD FX-8350 @ 4.5GHz, 16GB 1866MHz DDR3, nVidia GeForce GTX 660 2GB (non-Ti). There were noticeable frame rate drops near the end of all physics tests, but other than that, everything appeared to run smoothly, although I wasn’t running anything that measured frame rates.

  5. Indy

    November 7, 2014 at 11:24 am / 

    So, any chances for release of benchmark’s sources, now, when Unity 5 beta is more open?

  6. sama.van

    October 30, 2014 at 10:05 am / 

    FireFox 32.0.3 >> Windows 8.1, 64bits
    Error : ”Unable to find type [UnityEngine.dll]UnityEngine.AssemblyIsEditorAssembly”

  7. Moo

    October 15, 2014 at 2:15 am / 

    Chromium/Chrome is a lot faster than Firefox on Ubuntu 14.04. I got 20k in Firefox 32 vs 30k in Chromium 37.

    1. Moo

      November 28, 2014 at 2:22 pm / 

      Update! An extension was slowing down Firefox significantly. I disabled it and got 50k in Firefox.

  8. fffmalzbier

    October 14, 2014 at 5:48 pm / 

    On Chrome is working nicely but Firefox 31 (i know its outdated) gets stuck on the first test.

  9. pSupaNova

    October 12, 2014 at 3:44 pm / 

    Very interesting.
    Got 6103 on Nvidia Shield Tablet using Firefox. Ran all the benchmarks too.

    These javascripts hooks does it mean we could make a web app with say Backbone and have Unity running in a small canvas/window?

  10. Indy

    October 12, 2014 at 12:31 pm / 

    In all tests that require more pure WebGL performance than JavaScript calculations, Chrome beats Firefox. From years I have much more FPS in WebGL in Chrome than FF. I hope they will work also on that problem. Although I checked FF 35 nightly and there is nothing better in that area.

    1. Indy

      October 12, 2014 at 2:34 pm / 

      OK, I solved that. For some reason WebGL was running inside plugin-container instead Firefox process. Now I have much, much better results.

      1. Jonas Echterhoff

        October 12, 2014 at 8:33 pm / 

        Yes that should be possible – WebGL is just a canvas, which you can integrate into any web page.

  11. Allan Rowntree

    October 11, 2014 at 6:59 pm / 

    It looks like Intel is working on bringing SIMD to Javascript https://01.org/blogs/tlcounts/2014/bringing-simd-javascript

    1. Jonas Echterhoff

      October 12, 2014 at 8:29 pm / 

      Indeed – we hope that this will get us closer to native performance in areas which benefit from SIMD, like Physics or Animation & Skinning.

  12. Jon

    October 9, 2014 at 12:29 pm / 

    Why would you do a benchmark like this an exclude the 2nd most popular browser? You may not like IE, but a lot of your users are going to be using it!

    1. Jonas Echterhoff

      October 9, 2014 at 1:08 pm / 

      It’s not about liking, it is just a matter of which browsers we currently support in Unity WebGL. IE lacks some features we need so we don’t support it yet.

      1. Jon

        October 9, 2014 at 11:49 pm / 

        What are you talking about? I just ran your benchmark on IE11 and it worked fine.

        1. Jonas Echterhoff

          October 10, 2014 at 10:01 am / 

          Indeed, the benchmark seems to work on IE, but it is not a supported browser target for playing Unity WebGL games due to lack of some needed functionality such as audio, so it is not yet relevant for the question of exporting Unity WebGL games. This is bound to change, of course, so it would be interesting to see some numbers (feel free to post them). My dev machine I ran these tests on is a Mac, so numbers would not compare to the other results anyways – and if i ran on a windows machine, i would not be able to compare Safari (which has better support at the moment), so it is kind of difficult to come up with numbers directly comparing all four browsers.

    2. David

      November 7, 2014 at 2:06 pm / 

      “it is not yet relevant for the question of exporting Unity WebGL games:”
      Naive response. You need to focus on end usage, end goals and seeing the bigger picture.
      Dismissing something as not relevant when it clearly would be when it comes to selling these games/apps to agencies and customers is just ‘taking your ball home, because some of the team are old.’
      Yes, IE has issues, but do the test, let us all share the results and work on it. Otherwise it’s pointless to our customer centric businesses and so will become a pointless technology. We were so hoping this would be a solve all solution for that flash v’s html5 situation that resides on the web (i.e. when currently making smaller web games and apps). Your a mac user, then install a new boot of windows. Don’t just ignore the problems. If your not going to see the bigger picture then all is lost, is it not?

  13. Fred

    October 8, 2014 at 7:32 pm / 

    *With more than one tab open they all cause FireFox to crash hard.
    On my POS Windows box:
    Not a fan of the controll scheme of angery robotos- and not a fan of no obvius way to pause the demmo or leave it nicely. Speed wise the first area ran fine 4 janky cores.

    On my POS Macbook, that’s a 09 era machine none of the demos load something in the pipeline just causes the entire machine to become unusuable. In both cases there both very finickicky if they’ll load much less be playable.

  14. BluePhaze

    October 8, 2014 at 6:21 pm / 

    Oddly enough, I ran this test in IE and in Chrome and unless I am missing something IE beat chrome pretty decently.

    Chrome scored 0 in some of the categories (using the latest build as of today 10/8/2014):

    Chrome: 13570
    IE (11): 16880

    Of note is that Chrome scored a 0 on the Mandelbrot and a 0 on the particles.

    The only issue IE showed was that it did not support the web Audio API.

  15. Hervé

    October 8, 2014 at 3:35 pm / 

    Hello guys,
    Great benchmarks, could you please indicate which renderer you are using? deferred? forward?
    thanks

    1. Jonas Echterhoff

      October 8, 2014 at 3:42 pm / 

      All benchmarks are in forward rendering (WebGL also supports deferred, though, but that was added very recently).

      1. Hervé

        October 8, 2014 at 8:03 pm / 

        Makes sense since deferred would need some webgl 2 extensions it seems. When you’ll test the deferred version it would be nice to compare scenes with many lights.
        thanks for the quite feedback!

        1. Jonas Echterhoff

          October 9, 2014 at 10:36 am / 

          Actually, deferred rendering is possible in WebGL 1.0. Unity 4.x-style two-pass deferred rendering works out of the box. Unity 5.x also supports single-pass deferred rendering, which requires the WEBGL_draw_buffers extension.

  16. Benoit Fouletier

    October 8, 2014 at 10:19 am / 

    Tried it on iPad (now that iOS 8 supports WebGL). I can’t tell if it’s working because oddly enough, it fails to download. The Unity loader showed up and I got maybe 20% progress on the red bar, but then it stopped (due to screen lock?) and now says “server doesn’t answer” (tried restarting Safari, no luck, tried Chrome, no luck).
    I can’t wait to get Unity games on a tablet through the web: maybe for the next Ludum Dare I’ll be able to browse through the entries on my couch ;) !

    1. schmosef

      October 9, 2014 at 12:39 pm / 

      I tried it on my Tegra3 based SurfaceRT and it works! It’s super slow. But it works.

      The overall score was 3109.

  17. Simon

    October 8, 2014 at 8:30 am / 

    How does it perform on gui code, is that a bottle neck?

    1. Jonas Echterhoff

      October 8, 2014 at 9:15 am / 

      While I did not benchmark UI, UI is heavy on draw calls (which are just as fast as native), and scripting (which is sometimes faster then native), so I would expect UI to perform close to native in webgl.

  18. Art Gabriel

    October 8, 2014 at 3:33 am / 

    Cant wait to publish to webGL :) is there any “preview publishing option” plugin we can test out? Thanks in advance :) also i am sure it will unity webGL published can do networking using peerJS – im working on one now using unitywebplugin but not yet done – will post and share once ready. More power!

    1. Jonas Echterhoff

      October 8, 2014 at 9:17 am / 

      Not yet, a preview will be available once Unity 5.0 pre-order beta access is available. I don’t know when that will be yet. It should be possible to integrate something like peerJS to Unity WebGL using a plugin.

  19. schmosef

    October 8, 2014 at 1:02 am / 

    Will this benchmark project be made open source?

    1. Jonas Echterhoff

      October 8, 2014 at 9:14 am / 

      I’m planning to release it once Unity 5.0 is available.

      1. Ricardo Castro

        October 31, 2014 at 6:51 pm / 

        Is the benchmark already available? If so, where can we find it?

  20. Dave

    October 7, 2014 at 9:45 pm / 

    Cool. Your fake screenshot button does raise a question for me, though: how will browser communication work with WebGL builds?

    1. Jonas Echterhoff

      October 8, 2014 at 9:13 am / 

      You can use the same mechanism as in the Unity web Player. Also, we will allow adding browser JavaScript code to your project, so you can call JS functions directly from script code.

  21. Indy

    October 7, 2014 at 6:06 pm / 

    1. Are these two demos from other post updated according to newest Unity’s versions?
    2. Could you export this benchmark to Unity Webplayer that we could also compare it’s performance?

    1. jonas echterhoff

      October 7, 2014 at 6:47 pm / 

      1. No, they are still based on older 5.0 alphas.
      2. No, not right now, as these benchmarks would require a 5.0 web player, which is not publicly available. You can check the numbers from a native build given in the blog post, of course, for some comparison – but those are from my personal machine.

      1. Indy

        October 7, 2014 at 6:55 pm / 

        So, why is there 5.0.0b7 webplayer’s release channel on revisions list? It is only hidden but it should properly works.

  22. Chris B

    October 7, 2014 at 4:21 pm / 

    Emscripten is so impressive. Great work with the port – awesome stuff!

  23. Stefan Maton

    October 7, 2014 at 4:09 pm / 

    Mandelbrot GPU under Firefox 32.xx renders completely green surface (thus a score of >2,000,000)…

    Otherwise very nice benchmark…

  24. Nathan Warden

    October 7, 2014 at 3:47 pm / 

    I knew I picked the right web browser. :)

    In all seriousness, it looks like WebGL is coming along well and I’m sure these scores will only come up from here since WebGL has just recently started really gaining traction.

  25. Marty

    October 7, 2014 at 3:43 pm / 

    Hi,
    I know, that Unity 5.0 (Unity WebGL) release date is still unknown, but it would be really great, if you tell us answer for my question:
    Will you release this version earlier than chrome cancels support for plug-ins?
    Because the games will be disable in this browser.

    Thanks

    1. Kyle

      October 7, 2014 at 5:49 pm / 

      To my knowledge plugins already disabled in Chrome, so Unity shoud move fast if it want to win the competitors.

      1. Marty

        October 7, 2014 at 6:26 pm / 

        Hi,
        it is only blocked, you can still allow it, but at the beginning of the year (new version of chrome) it will not possible to run at all.

    2. jonas echterhoff

      October 7, 2014 at 6:31 pm / 

      I cannot comment on release dates, but I hope that we will at the least ship the 5.0 beta to preorder customers by that time.

  26. jonas echterhoff

    October 7, 2014 at 3:15 pm / 

    @ DEFEKT: For 5.0, we will focus only on desktops with WebGL. Any success running on mobiles at this point is coincidental :) However, many mobile platforms are starting to support it, and it is only a matter of time until it becomes fast & useful enough that we can use it to give a good Unity experience in WebGL on mobiles.

  27. Defekt

    October 7, 2014 at 3:06 pm / 

    Will Unity webGL also be supported on mobile once more mobile browsers begin to support it?

  28. jonas echterhoff

    October 7, 2014 at 2:56 pm / 

    @ KURYLO3D: We don’t support Unity WebGL on IE yet.

  29. kurylo3d

    October 7, 2014 at 2:50 pm / 

    No internet explorer bench mark?

Comments are closed.