One of the most controversial discussions in software development is about the need of writing automated tests.
In fact, all the arguments are plausible: some developers believe that it creates more confidence about their code, fearless of coding new features, and better product quality, while others believe that it can add too much complexity in the application with false negatives or flaky tests, slowing the development cycle.
After all, what is the right answer? Well, like everything in software development, it depends.
Why do we do testing?
I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence — Kent Beck
We all know the importance of automated testing. General good advice is to start testing at the very beginning, with tests that catches potential errors and ensure that the basic functionality will not break when launching a new release or before coding the next feature set.
If for any reason a bug is exposed, you should replicate it with a “unit test” and then fix it, this way you can guarantee that the same issue will not happen again.
The more your tests resemble the way your software is used, the more confidence they can give you. — Kent C. Dodds
The reason we write tests should be always about gaining more confidence. The confidence that your code will interact with other parts of the application as intended. Writing tests should be done to make sure the different pieces fit together. After you know things are working you can consider writing tests to increase code coverage. You can then also proceed to add detailed tests to cover edge cases and various scenarios if needed.
Testing shapes
As the application grows, different kinds of tests become harder to maintain. Martin Fowler and Kent C. Dodds have documented some testing shapes, testing pyramid and testing trophy, that helps us figure out what to focus as we go about writing tests. These models tell us to dedicate more effort to the base layers which add more value and then move upwards on the layers that will cost more time.
Do not use these shapes as a rule. There is always a trade-off between speed and confidence, as Martin Fowler says:
The pyramid is based on the assumption that broad-stack tests are expensive, slow, and brittle compared to more focused tests, such as unit tests. While this is usually true, there are exceptions. If my high level tests are fast, reliable, and cheap to modify - then lower-level tests aren’t needed.
After several years of using Pyramid, Kent C. Dodds came up with Testing-Trophy. It defines a layer for static typing and linting tools like Typescript and ESLint. This layer covers the essential parts of the application, like the proper casting of data and syntactical errors. It leaves most of the responsibility on the integration layer, which focuses on testing components together:
Integration tests strike a great balance on the trade-offs between confidence and speed/expense. This is why it’s advisable to spend most (not all, mind you) of your effort there.
Conclusion
In this post, we covered different arguments to test your software. We explored where to focus your time and what kinds of tests to write guided by the testing shapes.
I hope it motivates you to write automated tests in your project and removes confusion around what and how much to test.
Further reading
- Just Say No to More End-to-End Tests
- The Practical Test Pyramid
- Write tests. Not too many. Mostly integration.