One of the first things I do when working on any Xcode project is set up code coverage. If the coverage shows a hole, I know that area is lacking unit tests.
(Be careful, the opposite isn’t true: Just because some code has been touched by unit test execution doesn’t mean it’s actually covered. If altering the behavior of the code causes a test to fail, then you know it’s covered.)
Many people use CoverStory, a a code coverage browser app written by my friend Dave MacLachlan. Others use gcovr to integrate code coverage into their Jenkins continuous integration. Me, I use lcov because it lets me exclude third-party libraries from the measurements before generating an HTML report.
When a project is built from the command-line with xcodebuild, it places build artifacts into a “build” folder, kind of like the old days. But I want to measure coverage of unit tests as I run them from Xcode itself. This complicates things because build artifacts go into some obscure DerivedData subfolder. I solve this by having the build process export some of the project’s environment variables to a file. I’ll show you where to get the shell scripts I use to do all this.
Download the code coverage tools
I have a GitHub repository called XcodeCoverage. You’ll probably want to make some per-project changes. So if you use git for your projects, think about how you want to manage your fork – branch – submodule workflow.
Place the XcodeCoverage folder in the same folder as your Xcode project.
Then download lcov-1.10. Place the lcov-1.10 folder inside the XcodeCoverage folder.
Also (if you haven’t done so already) go to Xcode Preferences, into Downloads, and install Command Line Tools to get Xcode’s coverage instrumentation.
Set up your project
First, I assume you’re using Xcode 4.5. If so, you’re in luck: It’s easy. (Code coverage used to be a little more complicated in earlier versions of Xcode, as Claus Broch describes in Code Coverage with Xcode 4.2 and Code Coverage and fopen$UNIX2003 Problems.)
The first steps are the same, regardless of what coverage tool you use. Select your project, and go into its Build Settings. (I like to make these changes at the project level so that they’ll apply to all targets.) Find the “Generate Test Coverage Files” setting. Spin down the disclosure triangle to reveal your configurations. Enable this settings for your Debug configuration:
Do the same for the “Instrument Program Flow” setting:
So far, so good. Now let’s add the extra step so my scripts can get the project’s environment variables.
Select your primary target, and go into its Build Phases. Click “Add Build Phase” at the bottom, selecting “Add Run Script”. Edit it so it runs XcodeCoverage/exportenv.sh:
This last step has tripped up some people. If you use OCUnit — where the the main target is executed with tests injected — make sure you add the script to your main target (your app or library), not your test target.
On the other hand, if you use a different testing framework that depends on a separate testing app, then add the script to your test app, not the main target.
Now we’re ready for the good stuff. Run your unit tests.
Open a Terminal window, and cd to your project’s XcodeCoverage folder. Once the tests complete, run my getcov script:
Things will fly by. Eventually, a browser window should open. Now you can see your code coverage!
Change test code, then re-run
We measure code coverage to find the holes in our tests. So now let’s say you’ve added some new tests, and want to measure the coverage. As long as you haven’t changed your production code, the quickest way is to clean out the coverage data and re-measure. To do this, run the cleancov script:
With the previous results zeroed out, now we can re-run the tests, then do getcov again.
Change production code, then re-run
Time has passed, and you’ve made changes to your production code. When you decide to measure code coverage again, you should clear out earlier build artifacts. (Leftovers confuse lcov.)
To do this, hold down the Option key in Xcode’s “Product” menu. Select “Clean Build Folder”.
Now you’re ready to re-run the tests, followed by getcov.
Modify the scripts
There are two places in the XcodeCoverage scripts you may want to modify for your project:
First: In the envcov.sh script, LCOV_INFO determines the name shown in the report. By default, the name is Coverage.info. But if you want to maintain coverage reports for multiple projects, it would be best to change this name to match the project name.
Second: If your project uses third-party libraries, you’ll probably want to exclude them from code coverage measurements. You can do so by editing the getcov script and changing the exclude_data() function.
What I do is put all third-party libraries into a folder named ThirdParty. Then I exclude the pattern "ThirdParty/*" using lcov’s --remove command.
Let me know
Let me know how it goes! Do you have any questions, about XcodeCoverage in particular, or about code coverage in general?