Search Unity

No software releases without bugs. And while our QA is working hard to find as many bugs as possible, some of them still go under the radar. It might be because it only happens in some very specific configurations – looking at you, double SLI Titan owner ;) – or because we were focusing on a more risky area, or just because we plainly missed it. This happens in most software companies. However, those bugs still reach you and in order to fix them we need your bug reports. (Of course, if they’re already in the issue tracker, it means we know about them.)

If you ever sent us a bug report and never got a reply, you might have wondered why. In this post I’ll try to explain why that happens and what we’re doing to try and fix this as well.

First of all, let’s look at how many bug reports you are sending us:

Each column in this chart is a month. As we can see, for almost two years (and actually, for quite a long while even before that) we have been hovering at around 5k bug reports every month. Now, to clarify, this is 5k times the bug reporter was used to send us a bug report each month, of which only a very small amount are actual bugs. We call these reports incidents; they become bugs once we confirm internally that they are bugs.

We have a student worker team located in Vilnius and Kaunas who work on verifying incidents on our end and either turning them into bugs, or looking for some other resolution (finding a duplicate bug, finding that the issue is actually not a bug, etc).

Each student takes one of these incidents and tries to reproduce it on their machine using all the information supplied. If they succeed, we have a new bug report (barring it being a duplicate). If they can’t reproduce it they will write back to you asking for more information.

Anyways, here’s how many incident reports we have handled monthly since the beginning of 2016:

As we can see, at the beginning of 2016 we barely processed 2k cases a month. Now we’re almost at 5k. This was achieved by growing the team to 45 people (there were 20 people a year ago, and only 5 people in 2013) and by changing processes of how we handle the cases.

Combining these two charts together, we can see the delta of incidents every month. We want this to be at 0 (or above 0, as we’d rather handle everything and some of the old ones too).

As you can see, over time we’ve been approaching the goal of handling every incident you send to us. The major breakthrough happened at the start of the summer. Wanting to make sure that we handled everything, we created a cutoff date, labeled Day 0, and we tried our best to handle everything which came since then.

The spikes in June and August happened because of some mass closings of old cases.

Everything was going great until September came around. You see a lot of students lowered their work hours since a new semester began and they need more time for their learning efforts. 500 cases piled up since then but we are actively working on them and bringing down the pile by 100 cases each week!

This is huge – once we have a complete hang of this process it means that you’ll be able to rely on us to give you a timely response to your bug reports.


Sign up for our beta newsletter

Help us make Unity even better and get a sneak peek of what’s coming up in future versions of Unity. Sign up for our beta newsletter to get notified when there is a new beta version available.

22 replies on “Handling your bug reports”

My gripe with all of this is that there should not be a bug in the system older than 90 days. Every bug should be assigned to a department, and that department needs to stop work on new features until their bugs are gone. There are definitely bugs that have persisted through many versions…they’ll be eligible for Social Security soon.

I have the problem with Unity quitting because of an unexpected error! What can I do to fix it? I’m trying to learn to program games so I need to use Unity!!!!!

Actualy im a “little bit” mad and angry after bug report QA “communication”. I send a complete bug report with project files, builds and video records, but its simple unbelievable that big company like Unity does not take their developers serious and using mindless bots to try solve the problem. Do you think is funny to get answers with random selected lines from the code and try to act that causes the bug? Hard to decide its ridiculous or just stupid :(
I think better option is send one auto reply and let developers wait for response from “real human” team, like sending bulls#its what does have nothing with the reported bug and get people angry.

Now we see, there is no bot, but from the firs miscommunications we start to feel that there is no human who want to understand the bug report :)

Sorry for all, who are waiting on bug fixes; Unity must be spending their time fixing my reports. :p But seriously, I have very positive experiences with my bug reports. Very fast replies, sometimes tested and tracked within 2 hours, sometimes fixed within a couple of days. Keep it going!

We, users, must learn how to report a bug a real Unity bug. For example making a copy of our project and taking out all the game object till we finish with one game object with few components that trigger the bug. This small scene that contains the bug is simple to achieve and submit for those as me that is not good at bug report. A Unity video in the tutorial in the learning section on how to report a bug and a link to it when Unity eventually crushes probably can help us to produce a better bug report.

Is it possible to split Unity in two? one fix main thread version with no bugs known and fixed all bugs and an optional add-on that include the modification of Unity with the new incoming feature [Patch release? or add on]. In this way, if that feature is giving problem I can take it out or turn it off waiting for an update. In this way, we do not roll back.

That’s actually kinda where we are expecting to get to with the forthcoming Unity Package Manager, combined with our internal modularisation effort. As more and more parts of the engine get split off so they can be delivered through packages, you’ll be increasingly able to ‘mix and match’ what you want to use – so you’d be able to start with a stable, long-term-supported baseline, and then upgrade specific packages to get the new features that you’re most interested in.

QA is most of the time processing my bug-reports in a timely manner, this really did improve over the last years. Thanks a lot to the QA team!

It seems to me, the bottleneck is more on the developer-side nowadays. They don’t seem to be able to catch up with the amount of issues anymore.

I’ve submitted various bug-reports (eg 837604, 839040) that are a big deal for me, but even after a year, there is no fix available.

Any plans to improve in that area, to enable developers to fix bugs faster?

It’s not the most glamorous position but it is so important. I think more updates like these are great. Stability affects everyone so these are truly the unsung heroes. The tester and the coders all deserve our thanks so thank you. Please keep it up, it makes a difference!

Good but not surprised you get loads of bug reports as it’s free software – why not prioritise reports from people who have paid licenses?

You also previously told me documentation bugs go through same process and are slow to fix – separate them from the software bugs and get them fixed more quickly

Please rethink your strategy of shipping new versions with known bugs. You introduced a bug in a patch release of 2017.1, and fixed it in a later patch release. No harm no foul there, these things happen. This was a very small bug (someone threw an error when a mesh was assigned to a skinnedmeshrenderer). Then several weeks later, you shipped 2017.2 with this bug! The answer was: We don’t include known bug fixes when we’re on a run up to a new release. This tells me that your release date is MORE IMPORTANT than shipping stable software. Now a major release has the bug, and I’ll be answering questions about it for the next year. Seriously – why ship with known bugs with known fixes? How can meeting an arbitrary release date matter that much?

I have to agree with that. Unity gets you 90% of the way to your target and then it bugs out and/or crashes and you can implement everything yourself. For my last contract it was the video player….

“No software releases without bugs.”

That quote from inside the Unity mothership sure goes a long way to explain the precipitous decline in the quality of Unity releases over the past few years.

Whilst I agree with the quote, I do think that the quality has dropped, the focus has shifted onto making new features away from ensuring the existing features are solid or as good as they can be. I feel like Unity should seriously consider going opensource and letting the community help pick up the slack.

I’d say that the quality has generally improved – certainly how crashy the editor is. 6 years ago when I first started using Unity it would pretty reliably crash a few times a day – now it’s very rare. There still are pretty major bugs that slip through though.

Unity certainly has a problem with adding new features rather than fixing existing ones. The terrain system for example has been neglected for many years and can be pretty unusable in some scenarios. I understand this code is really bad and everyone is scared of it, but that’s all the more reason to get on with fixing it!

That said, on the subject on the article itself, it’s great that they’re improving their responsiveness. Until recently I’ve generally felt that with the more obscure Unity bugs your options were either to patch the engine yourself or to pay for the exorbitant premium support packages as the chance of anyone answering and fixing an obscure bug in a timely manner was very small. I’m glad to hear that isn’t the case now.

Comments are closed.