Search Unity

This is a story about how we are using Unity Cloud Build internally and how it can make life easier for you, our users, as well. Read on to learn how we used to deal with large project testing and which awesome new possibilities are available now!

Once upon a time

During development of the massive Unity 5 release, and the extensive changes it entailed, we frequently ran into issues with importing and building projects. For Unity 5 we wanted a cleaner API, which in some cases meant we had to break backwards compatibility. Often, when importing an older project in Unity 5, we had to fix scripts manually. We were also hitting major bugs and regressions in graphics, physics and performance related areas.

Our testers do a very good job at making sure that projects import, build and run properly in every new build on all our supported platforms, but since we are constrained by time we usually used small projects (like AngryBots or Nightmares) to run these tests. These projects don’t cover many of Unity’s features – any of which might break in a new version – and they are nowhere close to the size and complexity of some of the projects developed by our users.

We were fortunate enough to have a few major studios share the full project folders for some of their completed games with us (for example, Republique) and we started manually importing and building these games on every new build during the beta and release candidate phases of development. We found and fixed many issues before any of our users would have been affected by them, but it was a tedious and time-consuming task.

This is how the testing process worked back then:

  1. Install the new Unity build on Windows and OSX.
  2. Open a large project and reimport it. Wait a long time and check back from time to time to see if it has finished (often the Script Updater dialog would require a confirmation prompt which meant even more waiting).
  3. Fix any scripts or other issues and build the game for Mac and Windows Standalone.
  4. Switch platform to Android. Wait a long time again and check to see when it is finished.
  5. Run the game on Android.
  6. Switch platform to iOS. Wait again.
  7. Run the game on iOS.
  8. Repeat from steps 2-7 for all the other large projects we had (7 in total).

This was usually done by one person and it could take a few days.

The glorious present

We quickly started discussing how we could automate some of this. While we were busy figuring that out, in another part of the company work was underway for the official release of what is now Unity Cloud Build. Cloud Build seemed like a perfect fit for automating testing of these large projects.

Fast forward to the present day,  and after we released Unity 5.1, and Cloud Build has been out there for a while, testing large projects goes through an entirely different process (described below with pictures for your viewing pleasure):

1. Add project to Cloud Build

2. Press build for all supported platforms (currently Webplayer, Android and iOS with standalones and WebGl in the pipeline)

3_receive_notification

3. Receive an e-mail notification from Cloud Build when everything is finished

4. Share the build link with any/all testers, open the link in the browser, install the app, run and profit!

This saves us a tremendous amounts of time, since all it requires to make the build is a few clicks and all builds for all projects are done in parallel. As soon as the builds are ready we receive the notification e-mail, and testing can begin on all platforms. If anything fails during importing or building we also get a notification and we can act on it immediately.

Unity Cloud Build can also be configured to automatically poll a repository for changes and rebuild the projects automatically. It can rebuild projects on any supported Unity version.

A brighter future

Since we want to be able to scale up to testing more projects, the limiting factor now is running the builds on devices. The more projects we have in Unity Cloud Build, the more builds we have to install and run on devices.

The biggest problem with testing on mobile is device fragmentation (especially on Android) and we can only test on a few of the most popular models. We would like to know how these builds run on most devices (including some of the more esoteric ones). To that end, we are currently investigating services like TestDroid and AppThwack.

These services give us access to hundreds of devices and we can run our projects on any number of them. They offer a REST API that we can use to feed builds from Unity Cloud Build directly into them. What we get in return is performance data (CPU, Memory, Threading), screenshots of the game while running on the device, the ability to run custom testing scripts, get device logs and more.

By feeding all this data back into our own data warehouse, we can keep track of metrics across Unity versions, projects and devices and quickly pinpoint performance, rendering, and input issues.

Example test run results from our Doll Demo project on TestDroid

Unity Cloud Build and you

Unity Cloud Build is the solution of choice for us when it comes to removing all the time-consuming tasks and bottlenecks involved in importing projects to newer Unity builds, switching platforms and building projects. But we built Unity Cloud Build first and foremost with you, our users, in mind. If you are just getting started on a new Unity project, sign up for the free Cloud Build option and see how easy it is to have us do the heavy lifting for you and share the final build results with your entire team. If you are a veteran Unity user working on one or multiple projects, you will find something suitable for you in one of our other licensing options.

So, what are you waiting for? Give Unity Cloud Build a try! It might just be the best thing that happened to you since we introduced the Cancel Build button!

10 Comments

Subscribe to comments

Comments are closed.

  1. Andrew Slater

    June 30, 2015 at 6:46 am

    I have questions posted in the forums for Cloud Build, but they’ve yet to be answered. Are there any Unity people assigned to checking that forum?

    Thanks :D

    1. Elvis Alistar

      June 30, 2015 at 9:49 pm

      We are checking the dedicated Cloud section on the forums. We didn’t get the chance to answer each and every question. We will try to do better :-)

    2. Hey Andrew,

      which question do you refer too?

  2. I wish that Cloud Build would be available for Win/Mac/Linux stand alone builds…

    Oh, and I wish that the Cancel button would actually work. :-P

    1. Elvis Alistar

      June 30, 2015 at 9:44 pm

      As I have written in the blog post, we are working on adding support for Standalone player, WebGL and maybe more as fast as we can.

      If Cancel button isn’t working for you please submit a bug report on it and reply with its case number. I’ll take a look at it immediately.

  3. I enjoy using the Cloud Build and am looking forward to the cloud services as they continue to evolve. Are there plans to write an article later on regarding automated testing of a Unity game using one of the testing automation services? Would be interested in reading more detail on how it gets pulled into the mix and if Unity Test Tools are involved. Thanks!

    1. Elvis Alistar

      June 30, 2015 at 9:42 pm

      Glad to see you’re already using Cloud Build. And, yes, we have plans to write more about Unity Test Tools and how you can leverage that and maybe Cloud Build together to make sure you’re running automation for your game painlessly.

  4. Patrick Curry

    June 26, 2015 at 9:49 pm

    If bandwidth is a concern, you can download the resulting .ipa or .apk file from Cloud Build once, and then distribute it to your teammates via some other means, like a local LAN copy.

    For Xcode project manipulation, we recommend you use Unity’s Xcode API, which is well-supported in Cloud Build. The source code is shared here: https://bitbucket.org/Unity-Technologies/xcodeapi

    Cheers,
    Patrick

  5. The only problem i see with Unity Cloud is related with bandwidth.
    If your build has a size of 300mb (for example) and you have to download it in every single device, i am sure isn’t going to like to the IT guys.
    Another problem could be when you have to do some manual steps in your Xcode project, which usually is needed with some plugins. I think there isn’t any good solution for this, except trying to avoid plugins which requires manual integration :P.

    1. I usually end up with some extra Xcode steps, so this sounds like a bit of a limitation : /