Refactor tests, I say. Extract those helper methods. …But sometimes it's not that simple! You have be careful when extracting an assertion test helper. Especially when it becomes long or complex.
[This post is part of the series TDD Sample App: The Complete Collection …So Far]
Motivating Example: TDD Sample App
In the Marvel Browser project, we created a Request Model to encapsulate the parameters to our FetchCharactersMarvelService. We used Constructor Injection to begin TDDing the network request. Now we want to verify translating the Request Model into URL parameters.
First, we want namePrefix in the Request Model to become the nameStartsWith URL parameter. Here’s a naive test:
- (void)testFetchCharacters_WithNamePrefix_ShouldMakeDataTaskWithNameStartsWithParameter
{
QCOFetchCharactersRequestModel *requestModel =
[[QCOFetchCharactersRequestModel alloc] initWithNamePrefix:@"NAME" pageSize:10 offset:30];
NSURL *expectedURL = [NSURL URLWithString:
@"https://gateway.marvel.com/v1/public/characters?nameStartsWith=NAME"];
[sut fetchCharacters:requestModel];
[verify(mockSession) dataTaskWithURL:equalTo(expectedURL) completionHandler:anything()];
}There’s a problem: it tests for strict equality against an expected URL. What happens when we add more parameters? The test will fail, even though the new parameters aren’t relevant to this test!
We need to test only part of the URL argument. But how?
We could start by capturing the argument with the HCArgumentCaptor matcher from OCHamcrest:
HCArgumentCaptor *argument = [[HCArgumentCaptor alloc] init];
[verify(mockSession) dataTaskWithURL:(id)argument completionHandler:anything()];
NSURL *urlArgument = argument.value;Now we can examine urlArgument.query, which is the query string. We could search it for an expected substring:
NSRange range = [urlArgument.query rangeOfString:@"nameStartsWith=NAME"];
XCTAssertNotEqual(range.location, NSNotFound);But this doesn’t guard against accidental characters on either end of the string. No, it’s safest to use NSURLComponents to parse the URL into its parts:
NSURLComponents *urlComponents = [NSURLComponents componentsWithURL:urlArgument
resolvingAgainstBaseURL:NO];
NSArray<NSURLQueryItem *> *queryItems = urlComponents.queryItems;OK, now we need to search the queryItems array. We can create an expected NSURLQueryItem and test that it’s present in the array. Let’s put all the pieces together:
- (void)testFetchCharacters_WithNamePrefix_ShouldMakeDataTaskWithNameStartsWithParameter
{
QCOFetchCharactersRequestModel *requestModel =
[[QCOFetchCharactersRequestModel alloc] initWithNamePrefix:@"NAME" pageSize:10 offset:30];
NSURL *expectedURL = [NSURL URLWithString:
@"https://gateway.marvel.com/v1/public/characters?nameStartsWith=NAME"];
[sut fetchCharacters:requestModel];
HCArgumentCaptor *argument = [[HCArgumentCaptor alloc] init];
[verify(mockSession) dataTaskWithURL:(id)argument completionHandler:anything()];
NSURL *urlArgument = argument.value;
NSURLComponents *urlComponents = [NSURLComponents componentsWithURL:urlArgument
resolvingAgainstBaseURL:NO];
NSArray<NSURLQueryItem *> *queryItems = urlComponents.queryItems;
NSURLQueryItem *item = [[NSURLQueryItem alloc] initWithName:@"nameStartsWith" value:@"NAME"];
XCTAssertTrue([queryItems containsObject:item]);
}Wow, this is getting way too long and complicated!
Testing Test Code
This verification code is getting longer and longer. The longer it gets, the less confidence I have. And it doesn’t even have any control flow statements! What would happen to my confidence if we needed loops or conditionals?
Thankfully, there’s a good way out: Make it some kind of assertion test helper. Then test the helper.
In fact, depending on the approach you take, you can even TDD it. But some approaches are better than others…
Test Helper Approaches
There are several ways to write assertion test helpers. Let’s start with the most common way, and why you shouldn’t use it.
1. Helper Method that Asserts
I often see test helper methods that contain an assertion within the method. For example, here’s a test helper asserting that a URL contains a query:
- (void)assertThatURL:(NSURL *)URL
hasQueryWithName:(NSString *)name
value:(NSString *)value
{
NSURLComponents *urlComponents = [NSURLComponents componentsWithURL:URL
resolvingAgainstBaseURL:NO];
NSArray<NSURLQueryItem *> *queryItems = urlComponents.queryItems;
NSURLQueryItem *item = [[NSURLQueryItem alloc] initWithName:name value:value];
XCTAssertTrue([queryItems containsObject:item]);
}This looks like a straightforward application of Extract Method, right? The problem is with the XCTAssertTrue. When it fails, what location will it report? It’ll report the line number of the XCTAssertTrue. This isn’t helpful, because we want the line number where the test method invoked the test helper.
As more tests use this test helper, the situation gets worse. The reporting from Xcode becomes confusing. You have to dig around to find the actual tests that failed.
Test failures should point to the test code, not to the assertion test helper. Avoid putting assertions within methods or functions.
Test failures should point to test code, not the helper. Avoid putting assertions within methods.
2. Assertion Macros
XCTAssertTrue is able to report its own line number because it’s a preprocessor macro. It’s easy to see this and conclude that a good way to get the line number correct in an assertion test helper is to write their own macro.
Well, it depends.
Not long ago, Apple’s testing framework was OCUnit. In OCUnit, the assertion macros contained everything they needed within the macro. So it was common to copy this style:
#define MyAssert(containerObject, expectedValue) \
do { \
// Do something with containerObject \
// Do more things to get actualValue \
XCTAssertEqual(actualValue, (expectedValue)); \
} while(0);Line number reporting works because the preprocessor expands the entire macro in-place, including the embedded XCTAssertEqual. But writing multiline macros is a pain. Each line has to have the backslash continuation character. There’s the strange-looking do-while(0) trick to make the compiler treat those multiple lines as a single expression. There are preprocessor tricks like this that old hands may remember, but are easy to get subtly wrong.
I’ve written before about avoiding Xcode preprocessor macros. Let me add another reason: good luck trying to step through the code in a debugger.
Reason to avoid preprocessor macros: good luck trying to step through the code in a debugger.
A better approach is to minimize the macro, getting only what we need from the preprocessor: the file name and line number. And maybe self so you can identify the test class. Extract everything else to a function, passing these in as function arguments. This is the approach Apple finally adopted with XCTest.
Swift: Assertion methods are much better in Swift. You can treat the file name and line number as parameters with default values, which expand to the point-of-call. So there’s no need for macro magic! For more, see How to Make Specialized Test Assertions in Swift.
3. Predicate Methods
Instead of making a macro, it’s usually easier to create a method that contains everything but the assertion. Just return a boolean value. The method then becomes a predicate—converting its arguments into a single YES or NO value.
It’s easier to write tests for predicate methods. This also makes them easier to TDD.
The assertion then lives in the test body:
XCTAssertTrue([self doesURL:URL containQueryWithName:name value:value]);This is nice and concise. An assertion failure will report the correct line number. But what else will it report about the failure? Nothing.
You can improve this by having the assertion write out the actual URL:
XCTAssertTrue([self doesURL:URL containQueryWithName:name value:value],
@"Actual URL: %@", URL);Now a failure will yield the correct line, and it will also print the URL received.
But what if a predicate could do more than report YES or NO? What if it could report a diagnosis? …This is what “matchers” do.
4. Matchers
A “matcher” is a predicate. We configure the matcher when we create it. It can evaluate any object. And when the object doesn’t satisfy it, the matcher can report detailed information.
The testing frameworks Specta/Expecta, Cedar, and Quick/Nimble all offer matchers.
Hamcrest is a matcher framework with a unique DSL: matchers are designed to be composed from other matchers. This provides unparalleled control. Matchers can easily express only what they care about. This makes tests less fragile.
OCHamcrest matchers can easily express only what they care about. This makes tests less fragile.
Hamcrest is not only a library of matchers; it’s also a framework for writing your own matchers. So can we write a matcher to test that a URL contains a query item? See How to Make Your Own OCHamcrest Matcher for Powerful Asserts.
In the meantime, check out OCHamcrest and its cousin, SwiftHamcrest. They happily coexist with other testing frameworks.
Have you extracted an assertion test helper recently? Which approach did you use? Leave a comment below.
[This post is part of the series TDD Sample App: The Complete Collection …So Far]
