Assuring quality is a tricky task, and involves a lot of people — testers, developers, and even you the users! There are thousands of you and only a few of us, so today I want to talk to you about this:
If you see this Bug Reporter appear, odds are that you’ve just experienced a problem in Unity. I can honestly say that whatever it is, we want to fix it. As part of our QA processes, the test team (currently: me, Jens, Caitlyn, Rasmus, and Oleg) constantly review “the queue” of publicly submitted bugs in our QA/support database. We can do more with some bugs and less with others, and I want to help everyone understand how to tell us about their problem so we can actually fix it. When we review bugs, here’s what we think about:
What happened to this user and how did it happen?
Can I force Unity to do it again for me?
If we can answer both of these questions then we have enough information to investigate the issue and communicate our findings to you the user, or to a developer when discussing a fix for the issue. For this reason, the information you write in details 1) and 2) in the bug report is critical in helping us squash the bug forever.
Now, if we can’t understand what happened in the first place then there isn’t much we can do about the problem. So under detail 1) we ask that you thoroughly and completely describe the problem, using as much detail as you possibly can. The worst thing you can do here is write nothing. Writing anything you think is relevant to the problem is always better than writing nothing.
Once we understand the issue, we want to reproduce it. When we try to reproduce it, we perform a series of actions or “repro steps” to make the bug occur again. The first place we look for help determining the right set of repro steps is under detail 2). If the bug you’re reporting is frequently or consistently occurring, the best thing you can do is describe a discreet series of steps that we can take in order to reproduce the bug.
Sometimes a bug is specific to the scripts or assets in your project folder. In case it is, we ask that you always attach your project folder so we can make our test environment match your development environment as closely as possible. The bug reporter will automatically zip your folder and upload it, even if it’s as large as 500MB. And don’t worry, your project will only be used for testing purposes and it will never be shared outside of the Unity Tech office. If you can isolate the problem down to an extremely simple project folder that contains only the assets and objects relevant to the bug, even better! Either way, providing a project folder will greatly increase the likelihood that we can successfully reproduce the bug and fix it. But I’ll reiterate: if we cannot reproduce the bug, we cannot fix the bug. So pretty please with sugar on top, attach your project folder because we hate not fixing bugs.
I hope this has been an insightful look into our QA processes and you understand a bit more about the purpose of the bug reporter. To learn even more about bug reporting and the Unity QA processes, please watch the “How to Get Your Bugs Fixed” presentation video from Unite ’08.