I try to avoid asynchronous tests because they’re slow. But sometimes they’re worth having. When you do have them, it’s worth spending extra effort to make them as fast as possible.
My mistake was that I only focused on the speed of the happy path. Today, I learned a way to improve the speed of failure scenarios, at least around testing network requests.
Test speed matters a great deal. In Test Driven Development, you want to repeatedly take small, verified steps. Fast feedback is crucial to the process.
But even apart from TDD, test speed is important to keep your whole team moving forward. When someone submits a pull request, your build system builds your system and runs its tests. The time that requires stalls that particular pull request. And the more simultaneous pull requests you have, the worse things get.
Let me rephrase that in a positive way: small wins in test speed can accumulate into big wins for your entire team.
So let’s look at an asynchronous test that does real networking. If you have any tests like this, there may be an opportunity to improve their speed.[This post is part of the series TDD Sample App: The Complete Collection …So Far]
Before we start, a reminder: The kind of test we’re going to look at doesn’t belong in your unit test target.
And when I say “unit test,” I’m not particular about whether they’re actually testing isolated units. I use the term “unit test” quite loosely. What I really mean is “fast tests.” They can be testing complex systems. As long as they’re fast, I’m happy.
Asynchronous tests are fine, but belong in a separate test target. Even when they’re valuable, I don’t want asynchronous tests clogging up my TDD workflow.
I have code to authenticate calls to the Marvel Comics API. I want acceptance tests to verify that my code actually works. Here’s what I came up with for the Objective-C version:
This uses the OCHamcrest expression “assert with timeout (in seconds) that eventually (some value) will satisfy this predicate”. There are similar ways to express this in other test support libraries, from Kiwi to Quick & Nimble.
For the Swift version, I used XCTest’s built-in “expectation” API:
I don’t like the wordiness of the “expectation” approach. But because the Swift language is still a moving target, I’m being extra vigilant to avoid dependencies.
While trying to get away from this wordiness, I made a surprising discovery:
I’m not counting the first XCTFail which I added for completeness in the Swift version. Once the data task starts, any failure will occupy the entire time allowed.
My failure paths take 5 times as long as the happy path.
This happens in the OCHamcrest version. It even happens in the XCTestExpectation version, despite my liberal sprinkling of XCTFail statements.
In some asynchronous tests, there’s no clear trigger to complete the test. The loop will continue until the condition is satisfied, or a timeout occurs. This is the case when you’re trying to observe an indirect result — for example, observing changes in the UI.
But for a networking call, there’s a clear, unambiguous trigger. The code issues a network request. Whenever the completion handler is called, we’re done.
In a test, determining success or failure isn’t the responsibility of the completion handler. That comes later. The completion handler’s job is to move the test along to the verification phase.
Let’s first apply this to the XCTestExpectation approach. Originally, the expectation was:
And this expectation was fulfilled in the completion handler in the statement:
But now we want the expectation to be fulfilled in either success or failure. So really, the expectation is that the completion handler was invoked. The Act section of the test becomes:
Since the completion handler may be invoked with either a response or an error, we capture both. Then the Assert section of the test then becomes:
If I synthesize an error in the test code (such as using the wrong domain name), I get a description of the resulting responseError. If I synthesize an error in the production code (creating authentication parameters), I get a status code other than 200. I can look up status codes in the Marvel documentation to determine why the test failed.
Except for the network latency, everything is now significantly faster.
Applying that same logic back to the OCHamcrest version, here’s what we get for the Act and Assert sections:
An explicit trigger is introduced in the completionHandlerInvoked flag. It looks a little funny to see assertWithTimeout in the Act portion. But it’s now serving as the mechanism to re-synchronize an asynchronous test.
I introduce errors in both test code and production code to check the failure speed. It’s so much faster!
Let me summarize my new understanding. In any situation where a test is doing actual networking by providing a completion handler…
These are functionally equivalent, and slow for test failures. Instead, the job of a test-supplied completion handler is to:
Once the timeout loop has been escaped, the rest of the test uses the captured arguments to determine success or failure.
For a single slow test, the difference between 1 second and 5 seconds isn’t much. But if you have 100 of them…
Look through your asynchronous tests. Can you find any that should adopt this pattern?
Jon is a coach and consultant on iOS Clean Code (Test Driven Development, unit testing, refactoring, design). He’s been practicing TDD since 2001. You can learn more about his background, or see what services he can bring to your organization.