Who Tests the Tests? How to Increase Confidence in Your Tests 

Tests are code, too. So you may ask: who tests the tests? “Who watches the watchmen?” Is TDD flawed, because it's “turtles all the way down”?

This is about speed. The more I trust the tests over a section of code, the more fearless I can be with bold refactoring. Fearless refactoring means I don't spend time checking correctness. That's what the tests are for! Instead, I can focus completely on good design.

So how can you increase confidence in your test code?

Follow the Three Laws of TDD

I often mention the Three Steps of TDD, which I call “the TDD waltz.” But there are also the Three Laws of TDD:

1. Write no production code, except to pass a failing test.
2. Write only enough of a test to demonstrate a failure.
3. Write only enough production code to pass a failing test.

The Second Law keeps test code minimal. We write tests that contain no more than they need. If you haven’t seen the Three Laws in action, watch my 10-minute demonstration.

Write Short Tests

How long should a unit test be?

The ideal for which I strive is: three lines. Arrange, Act, Assert. The 3 A’s.

In practice, I separate those three sections with blank lines. This allows a little wiggle room for each section to grow slightly. But when any single section grows past three lines, I get an uncomfortable feeling. That is, if I have more than three lines in the Arrange, Act, or Assert sections, I pay close attention.

The discomfort leads me to ask two questions:

1. I’m getting feedback on how testable the code is. Can I simplify the design of the System Under Test?
2. Is it time to refactor the test?

But Doesn’t Test Refactoring Break the Rules?

The rule of refactoring is to start from green (tests passing) and end in green.

But your science mind is churning. How do you know the refactored test code still fails when it should? It seems too easy to weaken a test by mistake. For example, let’s say we delete an assertion. The test still passes! Isn’t this a recipe for failure?

The thing to understand is that almost all the time, refactoring tests is nothing more than Extract Method. And the method you extract is short—just a few lines. Nobody starts doing stupid things just for the sake of breaking TDD.

But unit testing is about confidence. Sometimes I still get a nagging feeling that asks, “Is this test still correct?”

Deliberately Break Something

There’s a simple way to test the tests: deliberately break something in production code. There are various ways to do this.

1. Delete a line.
2. Change a conditional to always true.
3. Change a conditional to always false.
4. Add a break statement to the beginning of a loop, so it never runs.
5. Add a break statement to the end of a loop, so it runs only once.
6. Change the return value.

Which should you choose? The one you have bad feelings about.

Doing any of these things should cause a test to fail. If not, it means one of three things:

1. If you deleted code, it wasn’t necessary.
2. A test is missing.
3. A test is broken, by not failing when it should.

Conclusion

Practicing TDD, writing short tests, keeping test code squeaky-clean… These things can help you move faster. Your increased confidence in the test code will help you refactor quickly and boldly. Skillful refactoring will gradually make your code easier to read, maintain, and extend.

When in doubt, break something and confirm whether your tests protect you.

But… what if the test code you need just isn’t simple? What if a test is too intricate? What if a test contains conditionals or loops? That’s when we need to write assertion test helpers, either in Objective-C or in Swift.

How confident are you in your current unit test suite? How will you increase that confidence? Leave a comment below.