Fast feedback is the chief benefit of TDD. Even if you’re not practicing TDD, anything that reduces the need to step through code saves a lot of time. For example, if you name your tests well, a test failure can tell you a lot just from the test’s name.
But what if you’re using a mock object? When a test fails, what can it tell you?
A hand-coded mock can tell you as much as you can code. But writing boilerplate gets old, so the reporting tends to be shallow.
And most mock object frameworks generate mocks that simply report, “Such-and-such was not invoked.”
This was also true of OCMockito. Until recently. Here are the 3 new descriptions of verification failures:
1. Report Incorrect Number of Matches
OCMockito verifies the number of times a particular method was called with matching arguments. This is most often done with verify for an implied count of 1. But you can also use verifyCount to match:
- An exact count
- At least a certain count
- At most a certain count
- Never
So far, that’s not new. What’s new is how OCMockito reports that a verify statement isn’t satisfied.
- If the number of matching calls is greater than the specified count, a call stack is reported for the first bad invocation.
- If the number of matching calls is less than the specified count, a call stack is reported for the last good invocation.
For example, let’s go back to the Marvel Browser project. We last left the Fetch Characters Service with this (incomplete) method:
- (void)fetchCharacters:(QCOFetchCharactersRequestModel *)requestModel
{
NSURL *url = [[NSURL alloc] initWithString:@"https://gateway.marvel.com"];
[self.session dataTaskWithURL:url completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
}];
}It’s incomplete, but what’s there is tested. Now let’s say we want to extract that last line to a new method. We extract it, but fail to remove the original line:
- (void)fetchCharacters:(QCOFetchCharactersRequestModel *)requestModel
{
NSURL *url = [[NSURL alloc] initWithString:@"https://gateway.marvel.com"];
[self.session dataTaskWithURL:url completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
}];
[self issueDataTaskWithWithURL:url];
}
- (void)issueDataTaskWithWithURL:(NSURL *)url
{
[self.session dataTaskWithURL:url completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
}];
}Now we get a test failure:
Wanted 1 time but was called 2 times. Undesired invocation:
MarvelBrowser -[QCOFetchCharactersMarvelService issueDataTaskWithWithURL:] + 129
MarvelBrowser -[QCOFetchCharactersMarvelService fetchCharacters:] + 228
OCMockito tells us how many times it was called. But in this case, it also gives us the stack trace of the first bad invocation!
And the stack trace is filtered, stopping right before the call stack gets into the test. This keeps the details of test infrastructure out of your way.
2. Report Method Called with Different Arguments
Here’s the test to check that we’re issuing the network request using HTTPS:
- (void)testFetchCharacters_ShouldMakeDataTaskWithSecureConnection
{
QCOFetchCharactersRequestModel *requestModel = [self dummyRequestModel];
[sut fetchCharacters:requestModel];
[verify(mockSession) dataTaskWithURL:hasProperty(@"scheme", @"https")
completionHandler:anything()];
}The method being verified takes two arguments. This test doesn’t care about the completion handler, so it accepts anything(). This is an OCHamcrest matcher.
The first argument is a URL. By default, method arguments are matched by testing for equality. We could construct a specific NSURL and use it as the expected value, testing for equality. But this would be an over-specified test.
As we TDD the construction of the URL to include parameters for the request, the actual URL will change. If we tested for equality (the default argument matcher), we’d have to update the test every time a new parameter was added. This is extremely fragile: the test would break at every unrelated change!
That’s why I specified
hasProperty(@"scheme", @"https")
as the argument matcher. The hasProperty matcher accesses the scheme property. The second argument is implicitly wrapped in an equalTo matcher. You may prefer to make this explicit:
hasProperty(@"scheme", equalTo(@"https"))
With all this in place, the test checks the URL to see if its scheme is “https”. What happens if the production code gets it wrong? What if it uses “http” instead?
Until recently, OCMockito would only report that the conditions hadn’t been met:
Expected 1 matching invocation, but received 0
I wrote this as a stopgap measure for the first release of OCMockito, always intending to beef it up. But it worked okay, so it stayed that way for several years!
So the method wasn’t invoked with matching arguments. Then what were the non-matching arguments?
Good news! As of v3.0.0, OCMockito reports:
- That arguments were different
- The expected arguments
- The actual arguments
- A description of each mismatched argument
- The call stack
Here’s what it looks like:
Argument(s) are different!
Wanted: dataTaskWithURL:an object with scheme "https" completionHandler:ANYTHING
Actual invocation has different arguments:
dataTaskWithURL:http://gateway.marvel.com completionHandler:<__NSGlobalBlock__: 0x1003972d0>Mismatch in 1st argument. Expected an object with scheme "https", but scheme was "http" on <http://gateway.marvel.com>
MarvelBrowser -[QCOFetchCharactersMarvelService fetchCharacters:] + 177
Can you see how this would cut down on debugger use? Especially if you got this report from your Continuous Integration system!
3. Report Method Not Called At All
What if the expected method simply wasn’t called? It depends. If no methods were invoked on the mock at all, that’s interesting information:
Wanted but not invoked:
dataTaskWithURL:an object with host "gateway.marvel.com" completionHandler:ANYTHING
Actually, there were zero interactions with this mock.
This happened during normal TDD process. I saw this message when the first test successfully ran, and successfully failed.
Now what if I try implementing something but call the wrong method? For example, maybe I call downloadTaskWithURL:completionHandler: by mistake. Then OCMockito reports all other invocations made to the mock:
Wanted but not invoked:
dataTaskWithURL:an object with host "gateway.marvel.com" completionHandler:ANYTHING
However, there were other interactions with this mock (✓ means already verified):
downloadTaskWithURL:https://gateway.marvel.com completionHandler:<__NSGlobalBlock__: 0x104a592d0>
MarvelBrowser -[QCOFetchCharactersMarvelService fetchCharacters:] + 177
Again, can you see how this would save time?
Conclusion
OCMockito used to have bare-bones reporting: “Expected 1 matching invocation, but received 0”. This was enough to keep me going for a long time. That’s because TDD works in such a tight cycle: when something goes wrong, it’s the last change you made.
But mock objects can complain in a variety of situations, not just during TDD. Whenever we use the debugger to step through a test, we should ask if we can get the information more efficiently. (For example, that’s a good reason to turn Xcode warnings up as far as you can stand them.)
So I hope OCMockito’s new reporting system serves you well! Anything that helps us code faster is a win that keeps paying.
Check out my new Tools page for OCMockito and other tools that can help you code more efficiently.
Anything that helps us code faster is a win that keeps paying.
