Since 2001, I’ve relied on an understanding of test execution flow in xUnit frameworks. But somewhere along the way, my understanding of the XCTestCase life cycle got messed up. I picked up an assumption that’s just wrong.
At best, it’s an assumption that can bloat our test runs. At worst, it can wreak havoc.
This is the big question. I had it right for a long time, but then I lost it.
You see, when Objective-C moved away from manual memory management, we no longer had to
release our objects. ARC is magical. Except for retain cycles, we got used to thinking that objects disappeared on their own.
Maybe I got lazy at that point. But I think it affected others, too. When people started writing test cases in Swift, I saw a common question:
“Why bother with
Why bother, indeed? For example, when creating your test fixture, what’s wrong with this way of making the System Under Test (SUT):
Isn’t this more “Swifty”?
I resisted this trend. No, I said, create your System Under Test in
setUp, and release it in
tearDown. Like this:
Why? I wanted to improve reporting of object life cycle problems. I’ve written xUnit frameworks before. So I knew that the control flow went something like this:
I want the creation and destruction of the SUT to fall within the try-catch scope. That way, if an exception is thrown, XCTest will report it as a failure of a specific test.
Even if it’s a crash, I can guarantee that the test log will show which test had started but not completed.
I thought I had it nailed. My results were good. But my understanding was incomplete…
Then one day, Benjamin Encz startled me with this tweet:
TIL: XCTest creates a new `XCTestCase` instance for every individual test invocation but doesn't `deinit` any of them after completion.
— Benjamin Encz (@benjaminencz) July 28, 2016
Wait, what? The XCTestCase is never really deallocated?
I ran an experiment and confirmed Benjamin’s observations. Wow, I thought, Apple screwed up XCTest in a big way.
…Until I realized I was wrong. I’d forgotten how xUnit frameworks are usually written. The question of deallocation was masking a different question:
When is an XCTestCase allocated?
The xUnit frameworks share a common architecture:
XCTest queries the runtime for all subclasses of XCTestCase. For each such class, it queries for all methods which
The assumption I mistakenly picked up was that to run a test,
This is wrong. I haven’t read the XCTest source, but if it’s anything like the old xUnit frameworks… This is what really happens:
In other words, it builds up the entire set of XCTestCase instances before running a single test.
tearDown were invented because the entire collection of test cases is created up front. They provide the hooks to manage object life cycle in tests.
This has two important implications:
tearDownwill continue to exist, even while other tests run.
Think of any object that alters global state, and shudder.
Let’s say you have an object that does some method swizzling when it’s created. It restores the original method when it’s deallocated. If you created this object in
setUp but didn’t release it in
tearDown, it will continue to exist. So the method swizzling done for one test will inadvertently affect all remaining tests!
Let’s boil this down to a rule-of-thumb:
Every object you create in
setUpshould be destroyed in
Remember, that’s not the only thing to do in tearDown. It’s there so we can reset things back to a clean state. What’s part of our global state? This includes:
These must all be cleaned up.
That’s also a frightening list of global state! Unit tests will do better by avoiding them if possible.
Have you been missing anything in tearDown? Let us know in the comments below, so we can all learn from each other.
Jon is a coach and consultant on iOS Clean Code (Test Driven Development, unit testing, refactoring, design). He’s been practicing TDD since 2001. You can learn more about his background, or see what services he can bring to your organization.