Unity QA status for 4.2
Our next upcoming release is Unity 4.2. Without actually going into what it’s going to contain in terms of features, let’s take a look at the other dimension we work along: Quality!
We just released 4.2 beta 2 this week and I will give you an honest status on the quality and bug situation in Unity.
Our rapid iteration for 4.2 took a hit because a lot of our developers attended GDC 2013 and we had some issues with an update of our build software.
As of writing this, we have 40 bugs to fix before we will be able to call it a release candidate, but we will naturally get some bugs during the beta phase we need to fix, so a reasonable call is 60 bugs left. How long it takes to get them done is a different matter on which I shall not speculate here.
It is good fun to take a look at how much we put into the release. From a QA perspective, this is one of the biggest bug fixing releases we have had.
We will hit 500 bugs fixed before 4.2 goes to final release. This includes 100 regressions from this or previous releases we have put in, so this release does bring an improvement in quality as well as features.
42% of all the bugs fixed have been found and reported by a customer, which we are deeply grateful for.
4.2 Customer bugs
Since 4.2 has been in alpha for quite a while, we have already had a good number of incidents (bug reports users submit, which we convert to bugs if we can reproduce them).
196 have been reported, 14 are still not processed and 119 have been reproduced and converted to bugs. Out of those 119, 30 have been fixed, so bugs which are not of a high priority are getting postponed in order for us to have as little code churn as possible. It should be some comfort that we are releasing faster if your bug has not been fixed this time around.
From a QA perspective it is always very interesting to see where the bugs are clustering, because clusters unfortunately indicate more issues to be found. The report I made for this highlights overall bug clusters and clusters specifically from user submissions, so we can see if there are patterns of bugs where QA is not doing well enough.
In this graph I see no immediate warnings, which is a sign that we haven’t introduced any big and buggy areas; at least not areas where users have reported many bugs to us yet.
Looking at the entire codebase, we have the following picture:
The numbers show priorities 1-7, where 1 and 2 are the ones slated for fixing in 4.2. Assets management lights up, but it is also a very big area. In total we have just over 2000 active bugs, which is not scary in a codebase the size of Unity’s, but we need to be vigilant about keeping this number down.
So that’s the current situation with our codebase. With all the work being done to solve the highest priority bugs I hope you will find 4.2 to be yet another improvement in our effort to give you the best and most productive tool for producing your game.