Hi, I’m Jon Reid, one of the maintainers of ApprovalTests.Swift. Today, we’re going to talk about legacy code. A lot of people want to add tests to their code, but the code already exists. And if the code is complex, adding tests seems even harder.
Characterization Tests
Disclosure: The book links below are affiliate links. If you buy anything, I earn a commission, at no extra cost to you.
Improve your test writing “Flow.”
Sign up to get my test-oriented code snippets.
A feature of ApprovalTests called “combination approvals” can help you bring legacy code under test. But before we get there, let’s review a concept from Michael Feathers’ book Working Effectively with Legacy Code.
This concept is called “characterization tests” and comes from the chapter, “I Need to Make a Change, but I Don’t Know What Tests to Write.” The thing about legacy code is that it’s already in use. Its current behavior is important, and we don’t want to change it unknowingly. So Feathers writes, “A characterization test is a test that characterizes the actual behavior of a piece of code.” Not “it should do this,” but “it currently does this.”
Think of a section of code as a somewhat opaque box. It’s not really a black box because we can look inside, but what we see may be messy or complicated, so we don’t know what it does.
With a characterization test, we say, “We may not know what the code does, but at least we know that if we give it this input, it gives us this output.” So we can add a characterization test to preserve that behavior. It shows that if we give this code the same inputs, we get the same outputs.
But of course, one input usually doesn’t do the trick. You want to use lots of inputs to cover various paths through the code. Once you’re throwing enough data at this code, you can feel safe about modifying that code, because the characterization tests will tell you if you inadvertently broke something.
A Regular Approval Test
Let’s see what this can look like in practice. Here we have a piece of code called MovieNight, which has a static function called movieRecommendations. It’s big, it’s ugly, it has lots of statements and nesting, and I don’t really know what it does. But we can call it from a test.
So let’s start off writing a simple approval test, try Approvals.verify, and just verify the output of calling MovieNight.movieRecommendations. It takes lots of parameters, so let’s make each of these as separate variables. So last name is Reid. Age is 3! Time available, 2.5 hours. And married… at 3 years old? Wow, no. Gender is male. And traditional title, nah, false. All right? So I’ll go ahead and plug these in, and fast-forward because it’s straightforward.
func test_movieRecommendations() throws {
let lastName = "Reid"
let age = 3
let timeAvailable = 2.5
let married = false
let gender = Gender.male
let traditionalTitle = false
try Approvals.verify(
MovieNight.movieRecommendations(
lastName: lastName,
age: age,
timeAvailable: timeAvailable,
married: married,
gender: gender,
traditionalTitle: traditionalTitle
)
)
}Okay. So now we have some input, and we can run this as a regular Approval Test. We can see that it gives us some output.
And it’s easy to ask the question, “Is this output correct?” Fortunately, we don’t even need to answer this question. This is legacy code that is in production: it is what it is. Unless I’m intentionally trying to change its behavior, its current behavior is correct. Therefore I can just approve this right away. And this creates a simple characterization test that locks part of the behavior. I can run this test, and it passes.
Now let’s turn on code coverage to see how much of the production code we touch with this one test. Run tests again. And we can see that there are some parts of this code that were touched, and quite a few that were not. And there’s this loop here where the coverage is questionable, because it did the innermost part just one time.
Combination Approvals with Just One
So what I want to do is put in a lot more cases. I could just create more test methods like the one we have. But better yet, instead of using regular Approvals, let’s call CombinationApprovals. And instead of verify, let’s call verifyAllCombinations. For verifyAllCombinations, we pass a function and the parameters separately. So let me rearrange the call to make these separate parameters. We now have a call to verifyAllCombinations that passes a function, but for the inputs it wants arrays, not single values. So let’s change these from single values to arrays containing those single values. And run tests again.
func test_movieRecommendations() throws {
let lastName = ["Reid"]
let age = [3]
let timeAvailable = [2.5]
let married = [false]
let gender = [Gender.male]
let traditionalTitle = [false]
try CombinationApprovals.verifyAllCombinations(
MovieNight.movieRecommendations,
lastName, age, timeAvailable, married, gender, traditionalTitle
)
}Now when I run this, I get very similar results. The only difference is that in addition to describing the output, it’s also going to describe the input. So you can see that for this input, it’s giving us this same output.
Combination Approvals with LOTS
So far, that may not seem like a useful change. But! The input values are now all arrays, so we can add a bunch more! I’ll keep the same last name, one last name. But I’d like to test the ages of 3, 10, 14, and 19. I’d like to test it with one hour, a little more than an hour and a half, two and a half hours, four and a half hours, and a whopping nine-hour marathon. Married is false and true. Gender, we’ll add female, and there’s another option, so we should specify non-binary as well. And another Boolean, so let’s send both false and true.
func test_movieRecommendations() throws {
let lastName = ["Reid"]
let age = [3, 10, 14, 19]
let timeAvailable = [1.0, 1.6, 2.5, 4.5, 9.0]
let married = [false, true]
let gender = [Gender.male, Gender.female, Gender.nonBinary]
let traditionalTitle = [false, true]
try CombinationApprovals.verifyAllCombinations(
MovieNight.movieRecommendations,
lastName, age, timeAvailable, married, gender, traditionalTitle
)
}And now it’s going to run all of these combinations, and with very little work, we’re going to get a lot more coverage. You can see that it’s now running 240 test cases!
Again, I don’t even have to look to see if it’s correct. I can just use the definition of legacy code to say, just preserve the current results. And now we have a nice test. Let me run this again. Test passes. And if we look at the test coverage, it looks like we may be covering every path. It took very little work to generate all of these situations and verify each result.
With very little work, we’re going to get a lot more coverage. You can see that it’s now running 240 test cases!
Now let’s make a simple mistake by adding an extra space. When we run tests, we’re going to get a failure, preventing us from unintentionally changing the behavior. And you saw how fast that was. This gives us a safety net we can use to refactor the code into something that’s easier to understand and maintain. Remember, “refactoring” means changing the design of the code without changing its behavior.
Review
So to review: You have a section of code that we’re not sure about and don’t feel comfortable changing. You start by throwing in a single input to get a single output. Then by using combination approvals, you can throw in lots of inputs to produce lots of outputs, by simply defining variations for each parameter. This will make it safer to change that code, redesigning it to be easier to work with.
If you have any questions about ApprovalTests.Swift, please ask in the comments below. You can also send a tweet to me on Twitter, @qcoding, and add the hashtag #ApprovalTests. Thanks for watching. (Please like, subscribe, and share.)
All Articles in this series
Watching the video it really does seem like magic! This may be the solution I need for a legacy code base. But, “legacy” means ObjC in my case. Just as you can write unit tests in Swift for ObjC classes and methods, is this also the same for Approvals.Swift?
It is a bummer that for iOS a python script has to be run to watch the test folder. Is that just a limitation on how XCTest works with the simulator?
Yes, as long as you bridge your Objective-C code over to Swift, you can call it from ApprovalTests.Swift.
The watcher is needed for iOS because the tests (running in the simulator) have no direct way to fire up your diff tool of choice (running on your Mac). So instead, the watcher runs on the Mac and waits to load up any diffs that come.
I always forget to run the watcher the first time. That’s why we added a reminder.
Thanks for your questions, Mark! Post any other questions as they come to you.
How would this run on a CI/CD server?
When a difference is found, two things happen:
– The received file is written to disk
– The received and approved files are passed to the reporter
By default, the reporter is a chain. It keeps looking for a diff tool it knows about. The end of the chain is ReportContentsWithXCTest which reports the mismatch using plain old XCTAssertEqual.
So on a CI server, chances are none of these other tools are in place. It will just report the failure to XCTest, which doesn’t require the file watcher. And you can add a step to your workflow to attach all files matching *.received.* to your job artifacts so you can access them from your browser. Then you can download them and diff them locally if you want the power of a diff tool to see the mismatch.