Static Analysis: Will It Free You from the Need for TDD? 

 June 14, 2016

by Jon Reid


There’s been a growing debate about static languages vs. dynamic languages. Writing my own thoughts on type safety helped me discover that I was unfairly judging Swift based on C++.

During the debate, another interesting article popped up: Eric Elliot's The Shocking Secret About Static Types. It points to a couple of studies. One study concludes that there is a lack of evidence that static typing reduces defect density. The other study concludes, “There is a small but significant relationship between language class and defects.”

Check out Eric's fascinating conclusion:

When it comes to bug reduction, I think it's fair to say: Static types are overrated. But when it comes to other features, static types are still cool, and may still be worth using. Bottom line: You want to reduce bugs? Use TDD. Use want useful code intelligence tools? Use static types.

We’ve come full circle: a discussion about static types has again brought us back to TDD (test-driven development). Why?

Current IDE Feedback

Right now, we get different levels of information from our IDEs.

  • They tell you if the code compiles.
  • They tell you if there are any warnings.
  • Static analysis goes further.

But more analysis is possible, beyond what Xcode gives us:

Please understand: I want every bit of feedback I can get. Even Xcode’s powerful static analyzer doesn’t go far enough for me. I want more!

Build the Perfect IDE

Imagine a perfect IDE. It unambiguously shows two states: red/green. It tells you if the code is working.

pedestrian red light and green light

It’s not possible, is it? I can use off-the-shelf tools, from Xcode to SwiftLint, to catch syntactic errors. But an off-the-shelf tool is never going to understand the semantics.

The only way to have my imaginary IDE is to have a custom tool that understands my domain requirements.

But it’s not imaginary. It’s what you get from test-driven development.

For those of you who don’t practice TDD: I understand the desire for that “small but significant” reduction in defect density. Hey, I work on projects with legacy code (that is, code without unit tests). When dealing with large swaths of legacy code, I want every tool I can bring to bear on the code. As Apple adds more warnings and static analysis to Xcode, I say, “Yes, please!” I’ll take every bit of feedback I can get.

And there are classes of bugs that TDD doesn’t address well. Like memory leaks.

But if you stop at static analysis, you’re selling yourself short. Build domain-specific dynamic analysis tools. They’re called microtests (or unit tests).

Here’s the thing: if you want to beef up your unit tests, doing so after the production code is hard work. It’s much, much easier to do them in parallel, with the test code leading the production code.

Welcome to test-driven development: dynamic analysis tools customized to your domain.

Are you ready to take a step further with microtests? Leave a comment below to share your journey.

Need Feedback Now? Quickly Hack a Spike Test

Jon Reid

Programming was fun when I was a kid. But working in Silicon Valley, I saw poor code led to fear, with real human costs. Looking for ways to make my life better, I learned about Extreme Programming, including unit testing, test-driven development (TDD), and refactoring. Programming became fun again! I've now been doing TDD in Apple environments for 20 years. I'm committed to software crafting as a discipline, hoping we can all reach greater effectiveness and joy. Now a coach with Industrial Logic!

  • {"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}