Search Unity

Today marks three years since our first patch release (4.3.7p1) way back on 15th of May 2014 and the subsequent creation of the Sustained Engineering (SE) team!

The SE team was established to help improve both the quality and the stability of the latest shipped (and some legacy) versions of Unity. Weekly patches and rolled up monthly releases are the main way we try to achieve this.

As of today we have released over 220 patches, which include almost 6000 fixes from multiple teams and areas of R&D.

The first incarnation of SE was a single Release Manager, with the sole objective of getting fixes to the users from time to time, while ensuring those fixes were not lost in the process. We are now a fully-fledged team of 15 developers and 3 QA engineers, actively fixing and improving the Unity codebase every day. The official launch event of the SE team was held in Brighton in October 2015 and the SE-QA team was formed in November 2016.

In addition to fixing bugs, the team owns some of Unity’s mature systems such as Kernel and the Version Control System. We also work on improving the user experience for our source code customers. One of the major focus areas for us right now is helping to reduce technical debt – more about this will be communicated in the near future.

As well as maintaining all released versions of Unity, members of the SE team also help to alleviate the pressure on other teams in R&D by handling backports, assisting with bug fixes and writing new automated tests. This helps those teams to concentrate on new features and their own high priority bug fixes.

Members of the SE team also visit Enterprise Support customers, focusing on improving the understanding of our processes and hence increasing confidence in our product. The team participates in both customer project reviews and upgrade tests for the new releases, and has a collection of customer projects that could be used for testing future releases.

We have come a long way, so let’s take a look at how we got here.

During Unity 5.0, the intention was to support a version for three months until the next major release. However, after user feedback we extended this to one year. This means we have multiple versions that we are supporting simultaneously, each version for a year since its initial release.

When possible we fix a bug on trunk first and then propagate that fix to each version, starting with the latest version and working towards the oldest – (a process known as ‘backporting’). This helps to ensure that you don’t see a bug get fixed in one version and then reappear when you upgrade. We also add new automated tests where possible, to reduce the risk of the problem being reintroduced in future versions.

During the last year, the quality of our patches became our priority; we improved our QA processes and scrutinised every fix that went in, sometimes rejecting fixes. When we first started patch releases, our process was to roll all the fixes into the patch and then QA would verify the reported bug fixes. We still verify each bug fix today, but we now have a dedicated SE QA team who also run a mini Release Acceptance Test (based on the areas touched, primarily) on every patch release. We also run a subset of our automated tests on every change we make, and the full collection of tests on each release.

We also don’t backport every fix; we recognise that each change carries an inherent risk and so we must decide if the benefit outweighs the risk. We do this by examining several factors including the user pain. The more stable a version becomes, the stricter we become on accepting changes.

It is interesting to note that we can now apply fixes, run our QA process and generate the release all within a week, which was considered an impossible goal when we started patches. In addition, we do this for multiple versions simultaneously, thanks to the dedication of all teams involved, developers, QA and more importantly the build and the tools teams.

Note: We are still supporting 5.4, 5.5 and 5.6 so these numbers will increase.

The Sustained Engineering team is continuing to invest in ways of improving the quality and stability of Unity even more and providing updates at regular intervals. We’re hard at work to see how much more we can achieve in the next three years – Happy Birthday to SE!

27 replies on “Happy Patch Day!”

I have recently learnt that Unity has separated teams for R&D and bug fixing. I don’t think this is a good idea for such a big product. I found an article about this:

The most important question here is: does the R&D teams aware of the bugs? Or better, do they know the mistakes they have made?

This is questionable since Unity is quite buggy. Even a so called “stable” (or final) version isn’t so stable. We often need to try different patches. Some bugs are just stupid. They can be easily encountered and reproduced in 100% rate. They should be fixed in the first place. Yet, they are in the released versions. I doubt it is because 1) the R&D teams don’t aware of the bug so they keep reintroducing it, and 2) the bug fixing team does not understand the core design of the product, so whenever a bug is fixed, other bugs are introduced.

I’m not against the idea of separating R&D and bug fixing, but I hope you can improve the process. Don’t make the mistakes mentioned in that article. Hopefully we will get a true stable version someday.
Have a good day.

Congrats are due to Unity! At the same time, somehow the process allows 100% reproducible bugs to remain for months on end. My iOS app sat stagnant for months as a result of one instance (basic glitch in text editing—probably took minutes to finally fix). The bug-fixing systems in place are clearly impressive and implemented by people who truly care, but I’m in the camp that fears new features are given too much weight when bugs can remain for so long. Or–something else is wrong. And it’s a years-long pattern. I love Unity, but I’m horrified to think that the ongoing bug situation may not improve! I hope it does.

I can’t help but think, “success hides problems”?

Why don’t you add fully customizable resolution for 2D projects. I think engine is missing that.

Great work.
One thing though, it used to be if you were a Pro user you got fast tracked response as to whether your bug was an issue or not, duplicate etc. Now I will often submit a major bug (like Editor frequently crashing on closing) and then get no response for months or indeed ever. Would be nice if some response was provided but I imagine you are overloaded

Stability and responses to bug reports improved very noticeably during these three years in my experience. Good job!

I can’t imagine Unity without the SE team anymore. Patch releases became a really important resource to me. Thank you SE team!

I wonder what the ratio between new bug-reports and bug-fixes are. I’ve the impression more bugs are reported than fixed.

I think your team has done a great job. It use to be I submitted bugs never to hear about the problem from Unity and now the past 3 or 4 bugs I’ve reported a Unity screener has collaborated with me to get all the details they need to replicate the bugs on there end in a couple of cases when my original bug report was ambiguous.

Funny seeing a link to my own bug report here. March 2013 so it’s been four years. Here’s my original submission in FogBugz:

Actually though I’ve seen a huge improvement in bugs getting fixed since the Sustained Engineering team was formed. I used to have a big list of bugs and everything’s been fixed now except that one. At one point I was planning to give a “fun” presentation in the style of that semi-famous “wat” talk about all the weird issues and gotchas in Unity 4, but before I got around to it just about all the stuff on my list I’d compiled got fixed! Seriously, well done Unity team, bug response has been great recently.

Comments are closed.