Categories & Tags
Archive

How we share Android test suites with the world

July 4, 2013 in Community News and Info, Technology by

In my last blog post I introduced you to our Runtime Test Framework and how it allows us to test the Unity Runtime API on a wide range of platforms. In this post I will tell you more about the test suite that we share with Google and others so they can integration test Unity with Android devices and the Android OS.

“What is our motivation for sharing this suite?” you might ask. It makes a lot of sense for us to push our test suites upstream and catch eventual problems as early as possible. We mitigate risk, in this case the risk that a certain device or a specific version of Android does not work with Unity. Also, the earlier we become aware of a problem the more time we will have to react and find a solution.

The Android Suite and Runner

The Unity Runtime Test Framework can generate a Unity project containing all tests, and then build an Android suite from it. We remove tests with external dependencies, e.g. WWW tests that require a running web server, but this is limited to a very small set of tests.

The external Android test runner is different from the one we use internally at Unity, which is based on C#/Mono. Our test frameworks and runners are very modular so replacing one test runner with another is very easy. The Android runner is based on the Android testing framework, and it starts a test suite by launching it as an activity on the device. As all runtime assertions are done player-side, on the Android device, the result of a test can be determined by the runner by using logcat and a simple parser.

The Runtime Test Framework can build an Android test suite in a few minutes, and running a pre-built suite takes 5-10 minutes on most devices (currently more than 700 tests covering all major areas of Unity as shown in the figure below).

Updating and releasing test suites

We typically update and ship a new version of the Android runtime test suite before we ship a new version of Unity. We like our bugs to be found and fixed as early as possible, and the results achieved so far are very promising. We have found bugs and we have fixed them before they did much harm. We are currently looking into finding more potential players who can benefit from this suite.

In my next blog post, which I will publish in a few weeks, I will tell you about our Graphics Testing Framework. Stay clear of bugs until then.

Share this post

Comments (18)

Comments are closed.

Derrick Lambert
4 Jul 2013, 8:25 pm

Im pretty sure Unity 4.2 in the slowest release in the history of releases. What the heck takes you so long? Extra lunch breaks recently?

Indy
4 Jul 2013, 9:31 pm

@up
Probably so many new exporters.
But must they all be in 4.2? What about Short Cycle Development and statement that if stuff is ready it comes in, otherwise it goes to next release.

Nathali Abbortini
4 Jul 2013, 11:21 pm

But if Unity 4.2 will have new exporters,then that does not have to mean that Unity 4.2 will be the slowest release ever.
So what are you actually talking about ?

Liszto
4 Jul 2013, 11:59 pm

I just hope an update of Mono 2.8.2 (DERPI DERP) to 3.0 or 4.0 (xamarin is good !) to increase the Garbage collector performance pleaaaaaaassseeeee !

5 Jul 2013, 2:51 pm

Yeah, Actually everybody is badly waiting for Unity 4.2.

I wish everybody at UNITY best of luck for it :) , and hope you can make it happen soon.

Cheers to UNITY ;) ;) the great tool the great team.

-

https://www.facebook.com/SantaInTheValleyOfGifts

Nathali Abbortini
5 Jul 2013, 9:40 pm

I remember that Unity was/is working on a new GUI system.
How far is that finished ?
Is it still going to be available in a future version of Unity 3D maybe ?

Jason
8 Jul 2013, 11:55 am

“Extra lunch breaks recently?”

almost… I think too many unity staff have a relaxed attitude and work when they feel like it. a lot of the programmers work on their own plugins instead of doing what they are supposed to.

If I was the director I would sack many of them. get some senior staff that have the balls to make them work, instead of trying to be their friends.

Senior staff at unity are wimps

8 Jul 2013, 2:18 pm

@Jason: you have no idea how hard people are working here on getting 4.2 out. Sacking them definitely would not help!

@Derrick: Unity 4.2 is far from being “the longest release ever”. That said, it is just around the corner.

Nathali Abbortini
8 Jul 2013, 2:40 pm

I remember that Unity was/is working on a new GUI system.
How far is that finished ?
Is it still going to be available in a future version of Unity 3D maybe ?.

8 Jul 2013, 3:34 pm

@Nathali “Jason” Abbortini: please go back to your cave, sir.

Kasper Andersen
8 Jul 2013, 3:54 pm

Yeah, @all, play nice!

Now, if you have some questions/comments relating to the test suites I will be more than happy to answer.

8 Jul 2013, 5:05 pm

I’m an alpha/beta Unity user and Unity staff is activelly working hard day/night, improving and fixing Unity! Even on weekends!
So please guys stop bashing.

8 Jul 2013, 5:44 pm

@Jason: While I sympathize with the frustrations of folks waiting for the new GUI system, it’s important to understand that it’s taking longer than we planned because we want to do it *right*. We could have, twice now, released Unity with a quick-fix GUI implementation. But we’re developing the next generation solution we expect to dominate the next ten years. So, we want to make a solution that will last and endure, and be utterly and entirely robust. We’re not afraid to take the extra time to do it right. The wait will be worth it. While you probably called our senior staff weak-willed, wimpy and accused them of ball-lackitude out of anger, the reality is they are some of the most visionary, hard-working, entrepreneurial people you’ll find.. and it’s because of their fortitude and dedication that Unity is as awesome as it is today.. :)

8 Jul 2013, 5:46 pm

BTW While you are waiting, and you need a solution in the meantime, we have a great selection of ingenious GUI tools on the Asset Store.. which you should buy. All of them. Right now. :)

https://www.assetstore.unity3d.com/#/category/141

8 Jul 2013, 9:57 pm

Unity 4 now has Mecanim right ?
The people that designed Mecanim,are they the same people that designed Motionbuilder ?

Sammy jurty
8 Jul 2013, 10:16 pm

wow that is good

9 Jul 2013, 1:31 pm

Bring it on soon guys. You always do work worth sharing. Cheers :)

26 Sep 2013, 8:41 am

Really it sounds nice. Lets see …..

Leave a Reply

Comments are closed.