Search Unity

A few weeks ago we talked about WebAssembly and its advantages over asm.js. As promised, now it’s time to look at the performance and load times of Unity WebGL in four major browsers.

It’s been a long time since we ran the Unity WebGL benchmark and published our findings. During this time, both Unity and the browser vendors released many new versions, added support for WebAssembly and implemented post-launch optimizations, especially during the past year or so.

On the Unity side, that means many changes have gone into the engine, both new features and optimizations, as well as WebGL2.0 graphics API support and an updated emscripten.

What to expect then? Given what we mentioned in the WebAssembly blog post, we expect Unity WebGL to perform better and load faster compared to the last time we ran the Benchmark using asm.js.

Here is how we run the tests

We rebuilt the Benchmark project with Unity 2018.2.5f1 using the following Unity WebGL Player Settings:

On WebAssembly, we take advantage of the automatic Heap growth feature described in the Wasm blog post, so we set the Memory Size to the minimum value. To measure asm.js we made a different build with a fixed Memory Size of 512 MB, which will be enough to run the benchmark. We changed linker target accordingly.

We tested four major browsers: Firefox 61, Chrome 70, Safari 11.1.2 and Edge 17. These are the latest stable releases at the time of writing this post, the only exception is Chrome 70 which is due to be released next month, containing a performance regression fix. We should mention that Firefox 62 also regressed in performance compared to Firefox 61 and we reported the issue to Mozilla.

Contrary to the last round of performance tests, we only tested desktop 64-bit browsers, for consistency of the results, and we used newer OS/HW:

  • Windows 10 – Intel Xeon W-2145, 32GB RAM, NVIDIA 1080
  • macOS 10.13.6 – 2018 MacBook Pro 15”, Radeon Pro 560X

To run the Benchmark on your machine, use this link (or as a zip). Note that depending on the browser version and os/hw you are testing on, performance may vary. On Windows, make sure to use a 64-bit browser.

Note that there is nothing preventing you from running the benchmark on your own mobile device (The alert box about Unity WebGL not supported on mobile devices has been disabled in this build).

Loading Times

One thing that changed since last time is that since Unity 5.6, Unity WebGL generates several unityweb files (Code, Data and JS Framework) that will be downloaded on startup, or fetched from browser IndexedDB cache when loading the same content again. This works pretty much the same in both WebAssembly and asm.js, however, you can expect loading Wasm code to be faster for the simple reason that the generated Wasm code is smaller. The Benchmark project outputs 4.6 MB compressed Wasm code as opposed to 6.1 MB for the asm.js version (data file is 5.6 MB and JS Framework file is ~87 KB).

Since network latency can affect the results, we measured Benchmark reloads (so that code and data were already in cache), and we served the build files locally. In addition, to speed up unityweb files load from IndexedDB, we changed cacheControl setting to immutable (default is must-revalidate). Here is how you can do the same for your own project html template:

This technique works well combined with Name Files As Hashes setting which makes Unity generate unique filenames.

WebAssembly vs asm.js

First, we are going to look at the total amount of time it takes to get to the main screen for both WebAssembly and asm.js (lower is better):

Findings:

  • Firefox is blazingly fast to load on both Windows and macOS
  • Both Chrome and Edge load massively faster when using WebAssembly
  • All browsers, except Safari, load faster with WebAssembly compared to asm.js.

In-Depth Load Times (WebAssembly-only)

Now let’s dive into the numbers that are relevant to WebAssembly. We are going to measure:

  • WebAssembly Instantiation: WebAssembly compilation and instantiation.
  • Engine Initialization: Unity engine initialization and first scene load.
  • Time to Screen: time it takes to render first frame.
  • Time to Interactive: time it takes to load and have a stable frame-rate.

Again, we are reloading the Benchmark so the unityweb files can be fetched from IndexedDB cache:

Findings:

  • Firefox is the fastest overall on both Windows and Mac
  • Edge performs really well. It’s interesting to see that it compiles Wasm really quickly (even faster than Firefox) but then is a bit slower to initialize Unity (Engine Initialization).

Tier-ed Compilation

As we can see, all browsers are faster to load when using WebAssembly compared to asm.js, but where does this improvement come from?

This is mainly due to the fact that they implemented a tiered compilation for WebAssembly. This means that the browser will now perform a very quick compilation pass at startup, then optimize hot functions later on.

Firefox shipped a tiered compiler with Firefox 58, back in January. Whereas Chrome shipped their new Liftoff compiler with Chrome 69. To give you a bit of perspective on this approach, let’s see what difference it makes in Chrome:

As we can see, the increase in engine initialization time is negligible, but the speed up in WebAssembly Instantiation is massive. This is great news since load times are critical for the web!

For more information about the tiering systems in browsers, check these blog posts:

Real-World Projects Load Times

Bear in mind that the Benchmark project doesn’t use a lot of assets and uses a small number of scripts. Both code and data files are relatively small, but real-world projects might result in larger builds which will impact the end-user’s experience.

Although loading times from cache are pretty fast now, don’t forget that you should still optimize your build size, so that first load time is reasonable. There won’t be a second load if the user drops-off while your content is loaded the first time!

We recommend checking the Optimizing Binary Deployment Size Unite Berlin talk, as well as the Building and running a WebGL project manual page.

Among the things that can also affect load times are shader compilation and audio decoding, so try to minimize that. The complexity of your shaders, as well as audio assets in your build can also lead to slower loading.

Performance

As explained in the first blog post, the benchmark consists of a collection of scenes that stress different parts of the Unity engine and produces a score based on the number of iterations that can be performed in a limited amount of time.

Last time, Firefox outperformed the other browsers. Let’s see what changed.

WebAssembly vs asm.js

Here is an overview of total scores using WebAssembly and asm.js (higher scores are better):

Findings:

 

  • All browsers perform better when using WebAssembly
  • On Windows, all browsers perform very similarly
  • On macOS, Firefox outperforms all other browsers. Notice that even the asm.js implementation is faster than other browsers WebAssembly implementation.
  • Safari is the browser that benefits the most by WebAssembly since it doesn’t support asm.js optimizations.

Scores Breakdown (WebAssembly-only)

Now, let’s take a look a the Individual Benchmark scores (scaled so that Chrome equals to 1):

Firefox is the fastest browser in nearly all benchmark scenes and excels in a few individual tests. However, if you measure Firefox 62, it will not perform as well because of the performance regression mentioned earlier, but we expect this problem to be fixed soon.

Note that WebGL2.0, a feature we haven’t benchmarked with before, is enabled by default in the build we used. So Chrome and Firefox use WebGL2.0, whereas Edge and Safari still use WebGL1.0. Having said that, we tried to disable it so that all browsers would use the same graphics API, but that didn’t seem to affect the results.

Outside of the context of a simple demo project, however, WebGL2.0 will result in reduced GC pressure and frame-rate will therefore be more stable.

For more information about performance in Unity WebGL, please check the WebGL performance considerations page in the manual.

Conclusions

The main takeaway is that today modern browsers load faster and perform better thanks to WebAssembly and that you can expect a more consistent user experience for your web content compared to asm.js. Having said that, we still recommend that you optimize your projects and test it on different browsers and os/hw.

In the future, we might update the benchmark project again so that it also stresses other areas, like ECS and the C# Job System as well as test with WebAssembly streaming instantiation/compilation and the upcoming multi-threading support.

We’re looking forward to hearing your feedback on the Unity WebGL Forum.

5 코멘트

코멘트 구독

코멘트를 달 수 없습니다.

  1. Seriously Unity needs to put up captcha or require login for the comments, the spam is so lazy 1990’s which doesn’t confer great trust in the technical leadership of the host.

  2. ifsc code of sbi vesu surat

    9월 21, 2018 9:10 오전

    A debt of gratitude is in order for sharing this data. I extremely like your post in particular.
    You have extremely shared a useful and intriguing post with individuals keep sharing
    thoughtfully take a gander at it http://iifsccode.com/banks/STATE_BANK_OF_INDIA/GUJARAT/SURAT/VESU/VESU

  3. Very interesting article. We have got the same tests with our app: Firefox is the best. Unfortunatly, the market is dominated by Chrome, and it’s hard to recommand some end users to change their browser. Anyway, the perfs are good enough, keep going Unity team !

  4. This is phenomenal news. I have been waiting for the reality of WebGL performance to catch up with the promise of run anywhere 3d gaming. It looks like WASM may be the key to it. I wonder if it is possible to create a PWA so the initial download only happens once. If so, that would be amazing.

    1. Hey Willindigo, I truly believe this is the future of gaming as well. Send me an email at alexstlouis@digiplaygaming.com, and we can chat more about it. I’m looking at implementing the games with a PWA as well. Really exciting stuff!