Search Unity

In my last blog post I introduced you to our Runtime Test Framework and how it allows us to test the Unity Runtime API on a wide range of platforms. In this post I will tell you more about the test suite that we share with Google and others so they can integration test Unity with Android devices and the Android OS.

“What is our motivation for sharing this suite?” you might ask. It makes a lot of sense for us to push our test suites upstream and catch eventual problems as early as possible. We mitigate risk, in this case the risk that a certain device or a specific version of Android does not work with Unity. Also, the earlier we become aware of a problem the more time we will have to react and find a solution.

The Android Suite and Runner

The Unity Runtime Test Framework can generate a Unity project containing all tests, and then build an Android suite from it. We remove tests with external dependencies, e.g. WWW tests that require a running web server, but this is limited to a very small set of tests.

The external Android test runner is different from the one we use internally at Unity, which is based on C#/Mono. Our test frameworks and runners are very modular so replacing one test runner with another is very easy. The Android runner is based on the Android testing framework, and it starts a test suite by launching it as an activity on the device. As all runtime assertions are done player-side, on the Android device, the result of a test can be determined by the runner by using logcat and a simple parser.

The Runtime Test Framework can build an Android test suite in a few minutes, and running a pre-built suite takes 5-10 minutes on most devices (currently more than 700 tests covering all major areas of Unity as shown in the figure below).

Updating and releasing test suites

We typically update and ship a new version of the Android runtime test suite before we ship a new version of Unity. We like our bugs to be found and fixed as early as possible, and the results achieved so far are very promising. We have found bugs and we have fixed them before they did much harm. We are currently looking into finding more potential players who can benefit from this suite.

In my next blog post, which I will publish in a few weeks, I will tell you about our Graphics Testing Framework. Stay clear of bugs until then.

18 replies on “How we share Android test suites with the world”

Unity 4 now has Mecanim right ?
The people that designed Mecanim,are they the same people that designed Motionbuilder ?

@Jason: While I sympathize with the frustrations of folks waiting for the new GUI system, it’s important to understand that it’s taking longer than we planned because we want to do it *right*. We could have, twice now, released Unity with a quick-fix GUI implementation. But we’re developing the next generation solution we expect to dominate the next ten years. So, we want to make a solution that will last and endure, and be utterly and entirely robust. We’re not afraid to take the extra time to do it right. The wait will be worth it. While you probably called our senior staff weak-willed, wimpy and accused them of ball-lackitude out of anger, the reality is they are some of the most visionary, hard-working, entrepreneurial people you’ll find.. and it’s because of their fortitude and dedication that Unity is as awesome as it is today.. :)

I’m an alpha/beta Unity user and Unity staff is activelly working hard day/night, improving and fixing Unity! Even on weekends!
So please guys stop bashing.

I remember that Unity was/is working on a new GUI system.
How far is that finished ?
Is it still going to be available in a future version of Unity 3D maybe ?.

@Jason: you have no idea how hard people are working here on getting 4.2 out. Sacking them definitely would not help!

@Derrick: Unity 4.2 is far from being “the longest release ever”. That said, it is just around the corner.

“Extra lunch breaks recently?”

almost… I think too many unity staff have a relaxed attitude and work when they feel like it. a lot of the programmers work on their own plugins instead of doing what they are supposed to.

If I was the director I would sack many of them. get some senior staff that have the balls to make them work, instead of trying to be their friends.

Senior staff at unity are wimps

I remember that Unity was/is working on a new GUI system.
How far is that finished ?
Is it still going to be available in a future version of Unity 3D maybe ?

I just hope an update of Mono 2.8.2 (DERPI DERP) to 3.0 or 4.0 (xamarin is good !) to increase the Garbage collector performance pleaaaaaaassseeeee !

But if Unity 4.2 will have new exporters,then that does not have to mean that Unity 4.2 will be the slowest release ever.
So what are you actually talking about ?

Probably so many new exporters.
But must they all be in 4.2? What about Short Cycle Development and statement that if stuff is ready it comes in, otherwise it goes to next release.

Im pretty sure Unity 4.2 in the slowest release in the history of releases. What the heck takes you so long? Extra lunch breaks recently?

Comments are closed.