Handling your bug reports

No software releases without bugs. And while our QA is working hard to find as many bugs as possible, some of them still go under the radar. It might be because it only happens in some very specific configurations – looking at you, double SLI Titan owner ;) – or because we were focusing on a more risky area, or just because we plainly missed it. This happens in most software companies. However, those bugs still reach you and in order to fix them we need your bug reports. (Of course, if they’re already in the issue tracker, it means we know about them.)

If you ever sent us a bug report and never got a reply, you might have wondered why. In this post I’ll try to explain why that happens and what we’re doing to try and fix this as well.

First of all, let’s look at how many bug reports you are sending us:

Each column in this chart is a month. As we can see, for almost two years (and actually, for quite a long while even before that) we have been hovering at around 5k bug reports every month. Now, to clarify, this is 5k times the bug reporter was used to send us a bug report each month, of which only a very small amount are actual bugs. We call these reports incidents; they become bugs once we confirm internally that they are bugs.

We have a student worker team located in Vilnius and Kaunas who work on verifying incidents on our end and either turning them into bugs, or looking for some other resolution (finding a duplicate bug, finding that the issue is actually not a bug, etc).

Each student takes one of these incidents and tries to reproduce it on their machine using all the information supplied. If they succeed, we have a new bug report (barring it being a duplicate). If they can’t reproduce it they will write back to you asking for more information.

Anyways, here’s how many incident reports we have handled monthly since the beginning of 2016:

As we can see, at the beginning of 2016 we barely processed 2k cases a month. Now we’re almost at 5k. This was achieved by growing the team to 45 people (there were 20 people a year ago, and only 5 people in 2013) and by changing processes of how we handle the cases.

Combining these two charts together, we can see the delta of incidents every month. We want this to be at 0 (or above 0, as we’d rather handle everything and some of the old ones too).

As you can see, over time we’ve been approaching the goal of handling every incident you send to us. The major breakthrough happened at the start of the summer. Wanting to make sure that we handled everything, we created a cutoff date, labeled Day 0, and we tried our best to handle everything which came since then.

The spikes in June and August happened because of some mass closings of old cases.

Everything was going great until September came around. You see a lot of students lowered their work hours since a new semester began and they need more time for their learning efforts. 500 cases piled up since then but we are actively working on them and bringing down the pile by 100 cases each week!

This is huge – once we have a complete hang of this process it means that you’ll be able to rely on us to give you a timely response to your bug reports.

Help us make Unity even better and get a sneak peek of what's coming up in future versions of Unity. Sign up for our beta newsletter to get notified when there is a new beta version available.