Unit testing part 1 – Unit tests by the book
If you are a developer, I assume you have heard about unit tests. Most of you probably even wrote one in your life. But how many of you have ever considered what make a unit test a good unit test? Unit test frameworks are just (fairly) simple runners that invoke a list of methods, one after another. But the code that is actually going to be executed in that method is none of the frameworks concern. How nice would it be to have a list of your favorite pizzas with an option of home delivery just by double clicking it. Why not to use a unit test framework to list all the options and save the time on writing GUI! Does it mean you have your test suite for making pizza orders?
A unit test framework is just a tool for writing tests but not every test written in this framework will be a (proper) unit test. Let’s paraphrase the pizza order example and imagine a system for ordering pizzas you want to test. The test case is supposed to validate the call is made the when I press “Make the order” button. The most straightforward solution would be to imitate all the steps in the test but you wouldn’t really want to receive a pizza every time you run a test, would you?
So what’s wrong with that test?
Let’s start with the scope of the test. A unit test, as the name suggest, should test a unit of work. Some people define a unit of work as a method but this definition is quite limiting and it’s better to see a unit of work as a single logical concept. The test shouldn’t actually communicate with the pizzeria and make the order. Instead, you would test if a proper order message goes out from the system once the button is pressed.
How do I verify that then?
Let’s assume the pizzeria has an online system for taking orders and our application has to send an HTTP request to order pizzas. As we don’t want to actually order that pizza, we could create our own server (a test double) that would simulate the behaviour of the original server, but without sending the real pizza. Such approach wouldn’t be that bad but it makes our unit tests depend on external resources and network communication. Additionally, how do we know that our mock server works as the original one. Or works at all? Shouldn’t we test the mock server? No! Instead of using the mock server, you should rather make the verification at lower level. You should validate that the object responsible for network communication would make the call to the server without actually making that call. By the call, I mean the HTTP request. The call in your code should happen, and it’s just the response that is faked (basically, the code in the tested module should not be different from the code in used in production). We want to run the test in memory, without using any resources. To achieve that we will need to mock some of the object using mocking frameworks or creating special implementations for testing purposes.
To decouple your modules use interfaces!
What if the payment is done by 3rd party system?
The payment system that connects us to a 3rd party website and expects us to type in your credit cards number is a true budget killer. We definitely want to skip that step but not only because we could end up with huge credit card debt, but we also want to save our own time by not typing in the credit card number every time the test is executed. We could of course ‘hire’ a student worker to do the job for us (and let him gain the invaluable experience) or we could be smart about it. A true unit test needs to be fully automated. No user interaction need to be required. We can simply achieve what we want by using mock objects again. Mocking lets us override behaviours of certain parts of the system which allows us to simplify and skip some steps in the ordering process. In this case we mock our Payment module and tell it to confirm our payment instantly, without redirecting us anywhere. Additionally, it gives us full control over the test. Imagine if the 3rd party servers are not responding or became super slow and all our test suites would fail because of that.
A simplified example of a test for making pizza order:
Great… anything else?
There are a few more things that will make your tests better that are not directly related to unit testing. Readability and maintainability will make it easy for the person that takes over the tests after you, to jump in and make the changes. Readable test can also serve as internal documentation for your feature. Less time spent on writing documentation gives you more time to write tests!
Never make your tests dependent on each other. The order of execution should never matter! That makes the tests hard to debug and maintain. If a test fails consequently after previous one did, simply because it was dependant on it, the true state of your code is obscured by the misleading results. Share the setup between tests, but always make the tests independent of each other.
Last but not least, you should take into consideration the execution time. Some people like to see the execution of unit test as a part of the compilation process. You don’t want the compilation to take too long so you should make your unit test run fast.
Unit tests are an important part of your general test suite and should be the ground base of all types of tests. To visualize the idea, take a look at the testing pyramid (inverted concept of the Ice-cream cone anti pattern) which depicts a healthy distribution of different types of tests in your test project.
So the original pizza ordering wasn’t that bad in the end?
Not necessarily. My point was to show what unit tests are and what they are not. An end-to-end scenario is also an option for an automated test but you shouldn’t base your test suites on them. The higher you go in the pyramide, the harder to maintain and debug the tests are and the slower they get. Integration tests and UI (thunder strikes) tests are also important but it’s all about balance.
It’s all nice in theory but you are probably thinking now how does it apply in Unity. Unity, to compensate the performance, has some limitation that works against testability. Lack of interface serialization is one of them. But not all of your code is required to be serialized! There are workarounds for those limitations I will write about in the future.
The next blogpost will be about designing your MonoBehaviour with testability in mind. Stay tuned!