April 15 2009Bug Reporting and You
Assuring quality is a tricky task, and involves a lot of people — testers, developers, and even you the users! There are thousands of you and only a few of us, so today I want to talk to you about this:
If you see this Bug Reporter appear, odds are that you’ve just experienced a problem in Unity. I can honestly say that whatever it is, we want to fix it. As part of our QA processes, the test team (currently: me, Jens, Caitlyn, Rasmus, and Oleg) constantly review “the queue” of publicly submitted bugs in our QA/support database. We can do more with some bugs and less with others, and I want to help everyone understand how to tell us about their problem so we can actually fix it. When we review bugs, here’s what we think about:
What happened to this user and how did it happen?
Can I force Unity to do it again for me?
If we can answer both of these questions then we have enough information to investigate the issue and communicate our findings to you the user, or to a developer when discussing a fix for the issue. For this reason, the information you write in details 1) and 2) in the bug report is critical in helping us squash the bug forever.
Now, if we can’t understand what happened in the first place then there isn’t much we can do about the problem. So under detail 1) we ask that you thoroughly and completely describe the problem, using as much detail as you possibly can. The worst thing you can do here is write nothing. Writing anything you think is relevant to the problem is always better than writing nothing.
Once we understand the issue, we want to reproduce it. When we try to reproduce it, we perform a series of actions or “repro steps” to make the bug occur again. The first place we look for help determining the right set of repro steps is under detail 2). If the bug you’re reporting is frequently or consistently occurring, the best thing you can do is describe a discreet series of steps that we can take in order to reproduce the bug.
Sometimes a bug is specific to the scripts or assets in your project folder. In case it is, we ask that you always attach your project folder so we can make our test environment match your development environment as closely as possible. The bug reporter will automatically zip your folder and upload it, even if it’s as large as 500MB. And don’t worry, your project will only be used for testing purposes and it will never be shared outside of the Unity Tech office. If you can isolate the problem down to an extremely simple project folder that contains only the assets and objects relevant to the bug, even better! Either way, providing a project folder will greatly increase the likelihood that we can successfully reproduce the bug and fix it. But I’ll reiterate: if we cannot reproduce the bug, we cannot fix the bug. So pretty please with sugar on top, attach your project folder because we hate not fixing bugs.
I hope this has been an insightful look into our QA processes and you understand a bit more about the purpose of the bug reporter. To learn even more about bug reporting and the Unity QA processes, please watch the “How to Get Your Bugs Fixed” presentation video from Unite ‘08.
laurent said on April 17th, 2009 at 9:30 pm:
when Unity crashes, does the bug report send you a dump of the memory, or any data that’s useful ? If not it maybe help the process.
Aras Pranckevičius said on April 17th, 2009 at 11:42 pm:
@laurent: it does send basid hardware information (OS version, CPU model, graphics card model etc.), the editor log files and in some cases the location of the crash (right now it does not send crash location on Windows because of technical issues).
ash said on July 18th, 2009 at 11:19 pm:
Would you also like reports on errors like null reference exceptions that pop up in the console from daily use?
angel said on August 18th, 2009 at 11:27 am:
Some bugs need to be solved quickly and the bug report system is a very long process until the bug, in the case it is really a bug, is fixed.I think the users prefer to post in the forum waiting for the solution of the problem, because you need to know if it is really an Unity bug and not your fault.
Graham said on November 10th, 2009 at 5:17 pm:
@angel – by all means use the forum to have a discussion with other users about the problem you have. If you really have hit a bug in Unity then getting it into our bug reporting tool is the best thing you can do to get our attention.
@ash – no, don’t report those. Typically they are errors in your project. Discussing them on the forum makes more sense. If a few eyes on the forum (and Unity employees do use the forum) suggest that what you have really is out of the usual, then the thread will probably conclude with “report a bug”.
a user said on January 5th, 2010 at 7:19 pm:
Where can I find the log files on different systems?
kevin said on February 23rd, 2010 at 9:33 am:
How can you tell who to put the blame on? We use unity differently than most people and the classes of bugs that we are getting are definitely not typically. As far as I can see most of our bugs are caused my multi-threading issues on windows. We even occasionally get C++ bugs.. “Pure Virtual function” which the crash reporter dialog doesn’t even come up. As an experience programmer without the appropriate information (debug symbols)I’m helpless in most of the cases as I can’t even find a work around until the bug is fixed
Samantha said on February 23rd, 2010 at 1:24 pm:
The log files can be found by clicking the “Editor Log” button in the Console of the Editor.
@kevin: If you are writing your own multi-threading, then any crashes you experience need to be fixed within your code. We suggest that only very experienced programmers attempt to use multi-threading, and do so at your own risk. In any case, please log your bug following the guidelines in the blog post and we will take a look.