One of the first things I do when working on any Xcode project is set up code coverage. If the coverage shows a hole, I know that area is lacking unit tests.
(Be careful, the opposite isn’t true: Just because some code has been touched by unit test execution doesn’t mean it’s actually covered. If altering the behavior of the code causes a test to fail, then you know it’s covered.)
Many people use CoverStory, a code coverage browser app written by my friend Dave MacLachlan. Others use gcovr to integrate code coverage into their Jenkins continuous integration. Me, I use lcov because it lets me exclude third-party libraries from the measurements before generating an HTML report.
When a project is built from the command-line with xcodebuild, it places build artifacts into a “build” folder, kind of like the old days. But I want to measure coverage of unit tests as I run them from Xcode itself. This complicates things because build artifacts go into some obscure DerivedData subfolder. I solve this by having the build process export some of the project’s environment variables to a file. I’ll show you where to get the shell scripts I use to do all this.
Download the Code Coverage Tools
I have a GitHub repository called XcodeCoverage. You’ll probably want to make some per-project changes. So if you use git for your projects, think about how you want to manage your fork - branch - submodule workflow.
Place the XcodeCoverage folder in the same folder as your Xcode project.
The easiest way to incorporate XcodeCoverage is via CocoaPods, as long as you don’t want to customize the exclusion rules.
Set Up Your Project
First, I assume you’re using Xcode 4.5. If so, you’re in luck: It’s easy. (Code coverage used to be a little more complicated in earlier versions of Xcode, as Claus Broch describes in Code Coverage with Xcode 4.2 and Code Coverage and fopen$UNIX2003 Problems.)
The first steps are the same, regardless of what coverage tool you use. Select your project, and go into its Build Settings. (I like to make these changes at the project level so that they’ll apply to all targets.) Find the “Generate Legacy Test Coverage Files” setting. Spin down the disclosure triangle to reveal your configurations. Enable this settings for your Debug configuration:
Instead of making these changes, you can incorporate XcodeCoverage.xcconfig.
Do the same for the “Instrument Program Flow” setting:
So far, so good. Now let’s add the extra step so my scripts can get the project’s environment variables.
Select your primary target, and go into its Build Phases. Click “Add Build Phase” at the bottom, selecting “Add Run Script”. Edit it so it runs XcodeCoverage/exportenv.sh:
This last step has tripped up some people. If you use OCUnit—where the the main target is executed with tests injected—make sure you add the script to your main target (your app or library), not your test target.
On the other hand, if you use a different testing framework that depends on a separate testing app, then add the script to your test app, not the main target.
Run!
Now we’re ready for the good stuff. Run your unit tests.
Open a Terminal window, and cd to your project’s XcodeCoverage folder. Once the tests complete, run my getcov script:
XcodeCoverage/getcov --showThings will fly by. Eventually, a browser window should open. Now you can see your code coverage!
Change Test Code, Then Re-run
We measure code coverage to find the holes in our tests. So now let’s say you’ve added some new tests, and want to measure the coverage. As long as you haven’t changed your production code, the quickest way is to clean out the coverage data and re-measure. To do this, run the cleancov script:
XcodeCoverage/cleancovWith the previous results zeroed out, now we can re-run the tests, then do getcov again.
Change Production Code, Then Re-run
Time has passed, and you’ve made changes to your production code. When you decide to measure code coverage again, you should clear out earlier build artifacts. (Leftovers confuse lcov.)
To do this, hold down the Option key in Xcode’s “Product” menu. Select ”Clean Build Folder”.
Now you’re ready to re-run the tests, followed by getcov.
Modify the Scripts
There are two places in the XcodeCoverage scripts you may want to modify for your project:
First: In the envcov.sh script, LCOV_INFO determines the name shown in the report. By default, the name is Coverage.info. But if you want to maintain coverage reports for multiple projects, it would be best to change this name to match the project name.
Second: If your project uses third-party libraries, you’ll probably want to exclude them from code coverage measurements. You can do so by editing the getcov script and changing the exclude_data() function.
What I do is put all third-party libraries into a folder named ThirdParty. Then I exclude the pattern "ThirdParty/*" using lcov’s --remove command.
Let Me Know
Let me know how it goes! Do you have any questions, about XcodeCoverage in particular, or about code coverage in general? Leave a comment below.
Was this article helpful? Subscribe today to get regular posts on iOS development.
Thanks for sharing this Jon, i’ve been banging my head against my desk for some time trying to simplify this task.
No matter how I try and set up the project though I get the following error;
ERROR: no .gcda files found in /Users/ which results in no trace, so no valid records.
I tried setting specifically at target level instead of porject level but it had the same effect. Have you ever seen this?
Thanks!
In order to get it working, I had to set the target specifically, project level settings didn’t work and if you’re running tests in the simulator, it must close before it generates files. Once that was all done it worked a treat. Thanks Jon.
Adam, I’m glad it was helpful! Two questions:
When you set it at the project level, and it didn’t take at the target level, was your target overriding those settings?
What do you mean “it must close before it generates files”?
Hmm, the path above got truncated, it wasn’t looking at users it was a valid path to the build folder in derived data :)
I tried generating the code coverage files from Xcode 4.5 with no luck.
My first attempt was with a C++ program that used Google Testing Framework and figured that the above instructions may only work with projects using Apple’s built in unit testing, so I created a small Cocoa app and used Apple’s built in Unit-Testing framework, but I still don’t get any output files.
I working on Mac OS 10.7, so I went over to my Mac OS 10.8 machine and created a Cocoa app there and again, I couldn’t find the files.
What may I be missing?
Thanks!
As another test, I made an iOS project with the same settings and still no luck.
Jaime, if you post your sample project somewhere, I’ll take a look.
Where shall I place it? Is there a mechanism on this site to post attachments? Thanks!
You can email me: jon [at] qualitycoding.org
Hi Jon,
Thanks for the great article!
This actually work with my Google C++ tests and it is a great addition to my development workflow!
Thanks again!
-Jaime
Glad to hear it, Jaime!
thanks for this post.
i got this error.
Found 30 data files in /Users/ignazioc/[…..]/Objects-normal/i386
Processing i386/AboutViewController.gcda
geninfo: ERROR: /Users/ignazioc/Desktop/[…..]/Objects-normal/i386/AboutViewController.gcno: could not open file
but “AboutViewController” has no test associated…maybe i need to have at least one test for class to run this tool?
thanks.
Ignazio, I work on legacy projects with many classes not covered by tests. So it must be something else. Try creating a sample project with some toy tests, then see if you can diagnose the difference.
Ignazio, did you resolve this error? I’ve encountered the same issue
I am trying to get this to work and got everything setup except when I run getcov, I get and error
geninfo: ERROR: need tool gcov!
Do I still need gcov to run this? I am running Moutain Lion with the latest Xcode.
Thanks!
Yes, download gcov-1.10. Then place the lcov-1.10 folder inside the XcodeCoverage folder.
I did download lcov, installed it into the XcodeCoverate folder and ran the script getcov but lcov seems to want gcov installed. Confused by your last replay since you said I did need gcov but you pointed to lcov.
Output from getcov
~/Library/Developer/Xcode/DerivedData/ContactManager-dxjozpbnyibstudkdiqnmmswcw
/Build/Products/Debug ~/Dev/ContactManager/XcodeCoverage
~/Dev/ContactManager/XcodeCoverage
Capturing coverage data from /Users/scottdensmore/Library/Developer/Xcode/DerivedData/
ContactManager-dxjozpbnyibstudkdiqnmmswcw/Build/Intermediates/
ContactManager.build/Debug/ContactManager.build/Objects-normal/i386
geninfo: ERROR: need tool gcov!
Reading tracefile Coverage.info
lcov: ERROR: cannot read file Coverage.info!
Reading tracefile Coverage.info
lcov: ERROR: cannot read file Coverage.info!
Reading data file Coverage.info
genhtml: ERROR: cannot read file Coverage.info!
The file /Users/scottdensmore/Library/Developer/Xcode/DerivedData/ContactManager-dxjozpbnyibstudkdiqnmmswcwyw/Build/Products/Debug/lcov/index.html does not exist.
Thanks,
Scott
Ohhh, sorry! To get gcov, go to Xcode Preferences and into the Downloads tab. Install Command Line Tools. I’ll add this to my instructions, thanks for staying on me.
No problem. I just realized that and was going to post it. I don’t install the tools anymore since most things work without them.
Found one more problem with the script. If you use NATIVE_ARCH you get some problems. I get i386 even though it should be x86_64. You should use CURRENT_ARCH. I found this discussion on the apple list : http://lists.apple.com/archives/xcode-users/2011/Jul/msg00086.html. I will send you a pull request with the change.
Merged. Many thanks!
Hi Jon,
Great post.
I am having issue setting up the Code Coverage for static library project.
I’m getting errors something like below:
Undefined symbols for architecture i386:
“_llvm_gcda_emit_arcs”, referenced from:
___llvm_gcov_writeout in libSomething.a(a.o)
I will not get this error if I disable ‘Instrument Program Flow’.
Any Idea?
Thanks
Hisyam, are you using an older version of Xcode?
I’m using the latest Xcode 4.5.2.
I have revisited my steps. It seems I have misconfigured the “Generate Test Coverage Files” and “Instrument Program Flow” settings.
I have redone the steps and everything works well now.
I am using a library project and getting the same errors
Undefined symbols for architecture i386:
“_llvm_gcda_emit_arcs”
….
Any clue? (on a new application project it works OK, so I assume all is correct)
To reproduce, create a library, setup the project , build is OK, runiing the unit tests generates the problem
I am using xcode 4.5.1
Same answers as elsewhere, Lior. Since it works on a brand-new project — with unit testing enabled — there’s a difference in your Build Settings. If you’re allowed to shared your project with me, I can take a look.
I am trying to implement code coverage with Kiwi Testing. I am not sure if you have ever worked with Kiwi but when I run the code coverage it seems to be returning me code coverage of the test files which is just 100% since the files are getting run all the way through. Is there any way to match the spec’s I’m writing to the files to get the code coverage of the actual file.
Thanks.
Matt, there’s no reason this shouldn’t work with Kiwi. Check the build settings at your target level and see if they were successfully applied.
I have figured out my error. I had your script running in my unit tests not my project it self. It is now working great. Thanks.
Ah! Makes sense, and good to know the symptom. Thanks for reporting back.
Hi, thank you for this great tool. However, do you know how I can get code coverage when the Test target does not compile the actual sources?
I have set up a test target, that holds a dependency on the main target and only has the test classes compiled in the “Compile sources” phase.
Sebastian, it sounds like you’re missing these steps: http://twobitlabs.com/2011/06/adding-ocunit-to-an-existing-ios-project-with-xcode-4/
No, that was not the problem. Actually, I had the same problem as Matt. Adding your script to my main target (and activating the Build settings for the global target) worked. Thank you very much for the script!
OK, that’s too many people stumbling at the same place. I’ve added a note warning people about this. Thanks!
Great post, John! First try with a *new* project in Xcode 4.5 and everything works as you described. However, when perform the same steps on an existing project of mine, no .gcda files end up being generated into the intermediates build folder, and the Coverage.info in the lcov folder stays at 0 bytes.
Would you know what causes this? It’s a library project, but that wouldn’t be the problem, right?
Here’s a link to the gist containing the terminal message and a link to the project on github: https://gist.github.com/c22e52d412d2fb22e457
Cheers, Eric-Paul – Amsterdam, the Netherlands
Hello to Amsterdam! Since it works on a new project but not yours, try to isolate the difference. I’d check a few things:
– When you look at Build Settings for your main app target, did “Generate Test Coverage Files” and “Instrument Program Flow” propagate, or are they overridden to NO?
– Is the exportenv.sh script specified on your main app target?
– Are you running Debug configuration?
I got the same error. Test Coverage Files and Instrument Program Flow are set t be YES. The exportenv.sh script is specified on my main app target.
I have no clue why no .gcda files end up being generated. Any ideas?
Nuo, what activity are you measuring? Is it unit tests? If so, which unit testing framework?
GHUnitIOS
I found those .gcda files in my testapp build. :s i was following your steps.
Good! It’s a little confusing with test apps that are separate from the production app.
Hi Jon – thanks for these tools! Testing with a simple project, using Xcode 4.6, in the LCOV – coverage report, AppDelegate.m, drilled in to the functions page, I get 0 Hit Count for “-[AppDelegate .cxx_destruct]”. What does that mean?
Eric, I’ve never seen cxx_destruct in my coverage results. Are you using Objective-C++?
This has just started showing up on a fresh project of mine as well. I’m not using Objective-C++. I am using Kiwi.
Hi Jon, It seems a great tool, but I get 0 hit count for any classes I’ve unit tested. It’s not an IOS project. I’ve set up the project as explained here. Does it work the same ? Where should I look to find errors ? Many thanks
Carol, it should work for Mac development, too. Find your DerivedData and navigate down through Build/Intermediates/app.build/Objects-normal. Next to your object files, do you see gcda and gcno files?
Thanks for the tutorial! I have everything up and running, but the code coverage seems wrong. I wrote some test using Specta but the coverage % is always 0%. Only app delegate is 100% and some of my first viewController class got about 3% coverage. I tried Clean project and ./clearcov several times but still no luck. Do you have any idea on this?
BTW, I also tried to read .gcda coverage files with CoverStory. But still got 0% coverage results. So it seems like tools are working correctly but I am missing something. :/
btw, I’m using XCode 4.6
I’m using XCode 4.6
I found the solution. ./getcov is reading coverage files from my main target, not the test target. I have to edit OBJECT_FILE_DIR_normal in env.sh to make it work.
Hello, I succeded to make it work by running the script on the test target instead of the app target. The proble was the same as Hlung. I also had the same issue with CoverStory. Now every thing works just fine. BTW I’m now using your test template, OHHamcrest and OCMockito. Thanks for those great tools !
Yeah, I ended up doing the same as you Carol :) Instead of fixing OBJECT_FILE_DIR_normal, moving the script to test target made it generate the correct path for reading the test target coverage files. BTW, I’m using Specta and Expecta to write tests.
Yes, I had written my instructions from an OCUnit perspective, where what gets executed is the actual app (injecting tests) instead of a separate testing target.
Carol, can you confirm that you’re not using OCUnit?
I’ll update the post to note this. Thanks!
Hi Jon,
I following your instructions but I get only .gcno files and no .gcda files. I’m using Xcode 4.6 and GHUnit 0.5.6. The tests run fine in simulator.
Have you any idea how I can fix this problem.
Thank you very much for the script!
GHUnit relies on a separate testing app, rather than injecting tests into the production app. So add the exportenv.sh script to the testing app, not the production app.
Hi Jon,
Thanks for such a great article!
I tried your steps and it works great on the XCode 4.5.1 version. I had a question however, should I set the Generate Test Coverage and Instrument Program Flow options for both Debug and Release versions, or should those settings be configured only for Debug? I guess these settings would probably come into effect only when you run the unit tests, and have no impact in the file size of the release binary. Is that correct?
Thanks,
Asheesh
Asheesh, I’m glad you found it useful.
Instrumentation changes the generated code. Don’t use it for shipping code. That’s why I do it for Debug only.
Hi Jon,
this post is really helpful.
I set up the code coverage for XCode 4.5 based on your instruction.
Thank you very much!!
When I generate output with lcov command, it has warning for all .gcno files ‘.gcno version ‘404*’, prefer ‘402*”. I searched in website, it’s caused by GCC version inconsistency.
But still I can get coverage data. will the data still accurate?
Regards
Yi
Yi,
Yes, the data is still accurate. The warnings are a nuisance, but harmless. Still, if anyone at Apple is reading… it would be nice to make this quiet…
How can I convert coverage.xml from gcovr to html report?
Sorry, I haven’t used gcovr so I don’t know. But there’s nothing stopping you from using gcovr for automation, then using XcodeCoverage manually on-demand. You certainly don’t need full HTML reports as often as automated reporting.
Thanks, works great. Can I use this in jenkins? I tried to convert output to cobertura xml, but without success
For Cobertura XML, use gcovr instead.
When I am entering ./getcov. I get the folllowng message:
No such file or directory
Also do I need to copy the Icov-1.10.
If so how would the shell script getcov be in the xcodecoverage directory because everything gets copied into the icove-1.10 sub folder.
Also do note I have installed the command line tools using xcode.
Thanks
Sanjeet
Sanjeet,
Yes, you need to download lcov-1.10 and place it within the XcodeCoverage folder. Use this link: http://downloads.sourceforge.net/ltp/lcov-1.10.tar.gz
As I said in mz earlier post
I had put icov1.10 in the XCodeCoverage folder and I get the following message when I tzpe the shell script ./getcov
No such file or directory
If I had that problem, I’d probably start adding echo statements to diagnose the shell script.
I get the following message:
envcov.sh: line 7: env.sh: No such file or directory
Ah! That’s more helpful. It’s not finding env.sh. You need to add a run script to execute XcodeCoverage/exportenv.sh. It’s the last step in the section “Set up your project”.
Currently I am not getting the dcda files generated.
Which testing framework are you using?
unittest++
I am having the same problem with OCUnit.
For OCUnit. I have added the shell script to export the project settings into the app target.
For unittest++ I am added the script to the test target.
Hmm. Strange. At this point, my advice is to set up sample applications using XcodeCoverage. See if you can get it working in the sample, apart from your real project.
Amazing job Jon! It worked like a charm.
Really simple and fast to configure.
Thanks a ton for sharing this!
It worked! Thanks for sharing a success story, Pablo :D
I have it working using OCUnit. unittest++ which is using c++ instead of objective c is still giving grief. Only the gcno files are being generated. Let me know if you have any ideas?
Hi Jon,
Thanks again for these articles, really useful. I have the code coverage up and running which is really cool to see. As I said in my comment on your UIViewController TDD post, I have been playing around with unit testing with Kiwi a little bit, but didn’t really know what was required for a full TDD implementation. Taking a look at my code coverage shows me that I have a lot of work to do!
I have a quick question, the coverage listing output is showing that I have no coverage on two functions that are not under my control; __copy_helper_block_ and __destroy_helper_block_. I believe this is probably to do with the way Kiwi is implemented, as I normally declare my SUT as a __block variable so that I can modify it within the Kiwi test blocks. Do you know of any way of ignoring the coverage for these two functions? I have taken a look inside your getcov script, and I see the exculde_data() function, I’m just not sure what to write inside that function so that these two items are ignored.
Any advice would be appreciated.
Sincerely,
Ben.
Ben, I’m puzzled. If everything is set up correctly, you should get coverage stats on your production code, not your test code. Why would any Kiwi get in there at all?
Hi Jon,
Just encountered this again, on second look I don’t think it is anything to do with Kiwi at all actually. In a particular piece of code I have declared some variables with the __block modifier like so:
__block NSString *variable1;
__block NSString *variable2;
I then subsequently enumerate an array using a block, and set these variables appropriately. Then when I view the coverage output, even though it looks as if I have 100% coverage in terms of each line being hit, it is only listed as 66.7%. If I then click on the functions link, I see these 4 lines are listed as not having been hit:
__Block_byref_object_copy_
__Block_byref_object_dispose_
__copy_helper_block_
__destroy_helper_block_
I realize it’s not really a big deal, as I can see that my code has been tested successfully, however it would be nice to be able to suppress these lines from lcov somehow just so the resulting coverage percentage is a little more accurate.
Thanks for your help!
Ben.
While lcov supports excluding entire files, I don’t know of a way to exclude particular functions.I guess you just have to shrug. But don’t forget to pat yourself on the back for tightening things up! Remember that the goal isn’t to satisfy lcov, it’s to enable you to refactor with high confidence.
It works great even here in Paris!
Thank you,
S.
You’re welcome, Simone!
Hi Jon, thanks for the article. I am able to get code coverage for my tests files but actually I want to know how much my STATIC libary code covered in unit tests. I have enabled “generate test coverage” and “instruments program flow” for static library which has generated .gcno file but not .gcda file in another folder. I don’t get coverage to my actual classes. Could you please let me know how to use coverstory and generate code coverage for static library? Thanks.
Hi Jon, I got this working. I just needed to add .m files in the tests target and nothing to do with static library actually. Very nice tool I would say. Very useful! Thanks again.
Great! Glad you figured it out.
I setup everything as outlined, run unit tests and then execute getcov command, but get the following output.
Any idea what I am doing wrong? Thanks in advance.
$ ./getcov
~/Library/Developer/Xcode/DerivedData/CodeCoverateTest-aoqsimcwfzfklcargqyfmdrjkyxe/Build/Products/Debug-iphonesimulator ~/Projects/iOS/CodeCoverateTest/XcodeCoverage
~/Projects/iOS/CodeCoverateTest/XcodeCoverage
Capturing coverage data from /Users/gmirabito/Library/Developer/Xcode/DerivedData/CodeCoverateTest-aoqsimcwfzfklcargqyfmdrjkyxe/Build/Intermediates/CodeCoverateTest.build/Debug-iphonesimulator/CodeCoverateTest.build/Objects-normal/i386
Found gcov version: 4.8.0
Scanning /Users/gmirabito/Library/Developer/Xcode/DerivedData/CodeCoverateTest-aoqsimcwfzfklcargqyfmdrjkyxe/Build/Intermediates/CodeCoverateTest.build/Debug-iphonesimulator/CodeCoverateTest.build/Objects-normal/i386 for .gcda files ...
Found 4 data files in /Users/gmirabito/Library/Developer/Xcode/DerivedData/CodeCoverateTest-aoqsimcwfzfklcargqyfmdrjkyxe/Build/Intermediates/CodeCoverateTest.build/Debug-iphonesimulator/CodeCoverateTest.build/Objects-normal/i386
Processing i386/AppDelegate.gcda
geninfo: Negative length at /Users/gmirabito/Projects/iOS/CodeCoverateTest/XcodeCoverage/lcov-1.10/bin/geninfo line 2413.
Reading tracefile Coverage.info
lcov: ERROR: no valid records found in tracefile Coverage.info
Reading tracefile Coverage.info
lcov: ERROR: no valid records found in tracefile Coverage.info
Reading data file Coverage.info
genhtml: ERROR: no valid records found in tracefile Coverage.info
The file /Users/gmirabito/Library/Developer/Xcode/DerivedData/CodeCoverateTest-aoqsimcwfzfklcargqyfmdrjkyxe/Build/Products/Debug-iphonesimulator/lcov/index.html does not exist.
Try double-checking the “Generate Test Coverage Files” and “Instrument Program Flow” settings at your application target level.
Hello Jon. When I check “instrument program flow” my app starts crashing. Without that flag my tests works fine. I have Xcode 4.6.2. Does anybody have this issue? And another one. If I add XcodeCoverage/exportenv.sh in main target it will not be called beause I have a logic test bundle.
Sorry, I had a bug which is hard to find: http://stackoverflow.com/questions/12418020/objc-retain-exc-bad-access.
Now all work perfectly. And I was right. If you want to create logic tests you should call XcodeCoverage/exportenv.sh in the unit test target.
Thanks for the great tool and the article.
Good to know, Alexey. Thanks for reporting back.
Hi Jon,
Thank you so much for making XcodeCoverage available publicly. I got code coverage working on my iOS app fairly easily. I am running OCUnit logic tests. However, though I am seeing an app crash very similar to Alexey’s in the iOS simulator when “instrument program flow” is enabled. I am able to work-around the problem by disabling “Instrument program flow” and I still seem to get the code coverage report in the browser. I tried a number of app fixes but they did not work. What are the implications of disabling “instrument program flow” ? I am intrigued that I still have the solution working despite disabling the flag.
Thanks once again.
regards,
amit
Amit, I really don’t know what the flag actually does. It’s in Apple’s instructions dating back to 2007. If it’s working for you without the flag, great. But I’d keep an eye on things, and also try it out on Xcode 5.
Hi Jon,
Thanks for the nice article!
Following your steps, after I run the unit tests in Xcode, I get the following error when trying to invoke gcov.
./getcov: line 13: syntax error near unexpected token `then’
./getcov: line 13: ` if [ -e lcov ]; then’
never mind…my issue with blank spaces
Is there something in my scripts that goes wrong when there are spaces?
Yeah they don’t like spaces in the path names (/git/project_a/ios/Project with Spaces for iOS):
./XcodeCoverage/getcov: line 9: /git/project_a/ios/Project: No such file or directory
Yes, I avoid white spaces in file paths because, ironically, they cause problems in many scripts. But XcodeCoverage has been updated to handle them.
Hi Jason
Did you find a way to come around this issue of spaces in the folder path other than avoiding spaces? Do let me know what i could change in the script. Thank you
Siddharthan, how hard would it be to eliminate spaces in your folder paths?
Hello Jon,
Thanks a lot for making XcodeCoverage tool publicly available. Really it’s a great help to get coverage report but I am facing one problem. I have some test cases and I want to find code coverage using the same. Now after running of tests and coverage tool, the report is not getting generated for the files which actually are getting used (here I mean static library). I can see only few header files (.hpp / .h) in the report and no other files (.cpp) can be seen. Do i need to add them in dependency list? If yes can you tell me how to do that? Or is there any other step through which I get all the file names listed in the report which were used by the test cases?
Thanks,
Ashutosh K.
Ashutosh, is this static library something you build yourself, then use in another target?
No, I am not building static library myself.
Then you can’t measure its coverage. And if it’s not code that you own, why would you care?
Hi,
Is there any way to change output to spreadsheet(xls) instead of htm.
Thanks,
Anil
Anil,
I haven’t tried it myself, but look into gcovr + Cobertura.
hi,
I tried using your tool however I don’t seem to generate the .gcda files when I tested my applications. Here is the output for the terminal:
ERROR: no .gcda files found in /Users/siddharthan64/Library/Developer/Xcode/DerivedData/MyApp-etdgihqgczfycohakrrolnzgxqzw/Build/Intermediates/MyApp.build/Debug-iphoneos/MyAppTests.build/Objects-normal/armv7!
Reading tracefile Coverage.info
lcov: ERROR: no valid records found in tracefile Coverage.info
Reading tracefile Coverage.info
lcov: ERROR: no valid records found in tracefile Coverage.info
Reading data file Coverage.info
And for the console on Xcode:
DerivedData/MyApp-etdgihqgczfycohakrrolnzgxqzw/Build/Intermediates/MyApp.build/Debug-iphoneos/MyApp.build/Objects-normal/armv7/main.gcda: cannot open
profiling:/Users/siddharthan64/Library/Developer/Xcode/DerivedData/MyApp-etdgihqgczfycohakrrolnzgxqzw/Build/Intermediates/MyApp.build/Debug-iphoneos/MyAppTests.build/Objects-normal/armv7/MyAppTests.gcda: cannot open
profiling:/Users/siddharthan64/Library/Developer/Xcode/DerivedData/MyApp-etdgihqgczfycohakrrolnzgxqzw/Build/Intermediates/MyApp.build/Debug-iphoneos/MyAppTests.build/Objects-normal/armv7/ViewController.gcda: cannot open
Am I missing something?
I found the mistake here. I’m supposed to run the test on a simulator and not on a device while testing. The tool is totally cool. I hope my question can help someone out there. Thank you Jon Reid for the CodeCoverage tool.
Siddharthan,
I’m glad you figured something out; thanks for sharing. (I never run unit tests on devices.)
Same here. On simulator everything works fine.
Thank you Siddharthan Asokan and Jon Reid.
Hello Jon,
If I need to add code coverage for a library file of the type ‘filename.a’, Shall I just add it to the compile sources of the Test target? Or is there any other tool that does this?
Siddharthan,
You would need to enable the coverage flags in the library’s project. Subprojects are a bit painful to add to coverage stats, but it can be done because lcov supports merging of coverage stats.
First of all, thank you Jon for your nice work. Have you managed to get coverage working in XCode5/iOS7?
I am getting ERROR: no valid records found in tracefile Coverage.info. I have checked and both the instrument program flow and the generate coverage flags are set and the exportenv.sh is added.
Xcode 3 required special voodoo for code coverage to work. With Xcode 4, things became much easier. But now with Xcode 5, it seems we’re back to needing voodoo: “You need to insert calls to __gcov_flush() to collect coverage data with the iOS simulator.” https://github.com/leroymattingly/XCode5gcovPatch is a workaround that inserts the call on every tearDown.
Thanks Jon!
I will try the patch. I also noticed that the coverage works ok when I use an iOS6 simulator from XCode 5.
Jon
I’m trying to use this approach while running scripted automated tests with Instruments. It appears the Coverage.info file is empty. Should this work with automated tests, or only with unit tests? If it doesn’t work with automated tests, any suggestions to determine coverage for these tests?
Bernard,
https://github.com/leroymattingly/XCode5gcovPatch includes a patch to cause UIAutomation tests to call __gcov_flush(), which Xcode 5 now requires for coverage.
jon, i’m confused how to get this patch to call __gcov_flush() for automation tests. what do I do with the uiapplication+GCovFlush file?
nvm, its working by just adding it into my project.
See my post https://qualitycoding.org/ios-7-code-coverage/ for an even better solution.
Hi guys,
This blog is useful. Thank you so much.
I compile my C++ code using GCC but then, CODE coverage is not supported by the LLVM-GCC 4.2 compiler in OSX. And thus am using CLang++ compiler, which generates the .gcno files along with the object files (.o)
Now when i link my code against – libprofile_rt.dylib
I get Linker Errors
Undefined symbols for architecture i386:
“_llvm_gcda_emit_arcs”, referenced from:
What could be the problem?
Should i not compile my C++ code using CLang++?
Or Should i link against some other library?
Any answers would be appreciated.
I solved this.
I was missing the following Linker Flags
-Wall -fprofile-arcs -ftest-coverage
Ah, good. Thanks for sharing the answer, Kamlesh.
Jon Reid, thanks for this very detailed and informative post. I succeeded implementing it but when I am trying ./getcov its giving coverage report only for 2 classes. I tried implementing some small test cases for other ViewControllers as well but coverage report is showing only for earlier 2 classes. Can you please help me with this. Thanks in advance.
BTW I am using Xcode 5 and used __gcov_flush() as mentioned to get it work.
I got it. I didn’t add the ViewControllers to the testing target. Now added to the testing targets and it’s working fine. Thank you.
If you’re using SenTestingKit or XCTest, you shouldn’t add production code to your test target.
Hi
Thanks for posting this.
I too followed the same steps and got the same error as coverage info not found, like many others above.
I may be doing something differently.
I’ve used bullseye with visual studio inthe past, so my interpretation maybe biased.
After enabling coverage and instrumentation for debug mode, added build phase, i simply run the app in the simulator, execute various use cases on the ui and finally exit the simulator. (that’s how I used to do with bullseye).
Now when I run your script, I get the error, no coverage info.
Will this work only on the unit Test target and not on main target?
Please let me know.
Thanks
It really depends on your testing framework. And for iOS 7 work, you need more: see https://qualitycoding.org/ios-7-code-coverage/
Hi,
Just to add something that I found.
For GHUnit, XcodeCoverage/exportenv.sh should be added to the Main Project(Project which is being tested) and not to the test project.
Make sure to add “Application does not run in background: Yes” to the info.plist file of the GHUnit test app(not to the main project).
Once tests are executed(i checked this on the simulator) Make sure to shut down the app by pressing the home button.
Only after shutting down the app as described above, do execute the getcov.
Thanks for adding these GHUnit tips.
Thanks Jon. This was very useful.
Thanks for the nice tool. I am able to successfully integrate/execute the same.
I have a query is there anything additional need to be done as currently code coverage displayed is only for test bundle and not for the main app.
Then something’s wrong, Varun. Re-read the instructions carefully, making sure you get your targets right.
Can someone post how to setup XCodeCodeCoverage from Scratch instead of all the patches? I downloaded the sample application under CodeCoverage patch for Xcode5 and it doesn’t see to produce code coverage.
It will be of great help if someone can post YouTube steps for setting up codecoverage and viewing reports.
Thanks in advance
Harish
See my post iOS 7 Code Coverage – Fixed!
Jon Reid,
This is an awesome tool and I’ve been using it for a couple of weeks now. Unfortunately, after upgrading to Xcode 5.1 this morning, I can’t get it to work any more. All I get now in terminal is “ERROR: no .da files found in XXXXXX” and “ERROR: no valid records found in tracefile Coverage.info”.
Am I missing something or did upgrading to Xcode 5.1 actually break something?!
Thanks!
Yeah, Xcode 5.1 broke my scripts, but things are better now. See Code Coverage Fixed for Xcode 5.1
Hi Jon,
Im trying to display the results of the code coverage in the post-action run time scripts for Tests. I have them running manually, but am trying to automate it as much as possible.
I am currently running,
“XcodeCoverage/exportenv.sh
bash XcodeCoverage/getcov”,
It doesnt complain or fail, but my browser isnt opening with the results. have you been able to get something similar up and running?
Cheers,
Ah! The latest update to XcodeCoverage should help you; see the “Bonus” section of Code Coverage Fixed for Xcode 5.1.
Michael, were you ever able to solve this?
Personally, I’d avoid showing coverage results after every test run, because with TDD, I run tests often. Like, on the order of every minute.
Hi Jon,
I tried using this tool on my code. The code consists of Java, webservices and C++.
Java is using web services for C++ business logic.
I put the Xcodecoverage in the C++ project package and enabled the Generate Test Coverage Files and Instrument program flow in Xcode as per the steps you mentioned then ran the scenario. It is generating gcno files but gcda files are not getting generated.
I want to ask you one question that whether in this flow(Java-webservices-C++) your tool can be used to generate the coverage report for C++ code?
Cheers
S
Shweta, XcodeCoverage works on the C-based side of things, so I think it should work on C++.
Hi,
I am also facing same problem encounter by Scott Densmore,
~/Desktop/CodeCoverage/20Aug/build/Coverage-iphonesimulator ~/Desktop/CodeCoverage/20Aug
~/Desktop/CodeCoverage/20Aug
Capturing coverage data from /Users/abc/Desktop/CodeCoverage/20Aug/build/sdkTest.build/Coverage-iphonesimulator/sdkTestTests.build/Objects-normal/x86_64
geninfo: ERROR: need tool /Users/abc/Desktop/CodeCoverage/20Aug/XcodeCoverage/llvm-cov-wrapper!
Reading tracefile Coverage.info
lcov: ERROR: cannot read file Coverage.info!
Reading tracefile Coverage.info
lcov: ERROR: cannot read file Coverage.info!
Reading data file Coverage.info
genhtml: ERROR: cannot read file Coverage.info!
The file /Users/abc/Desktop/CodeCoverage/20Aug/build/Coverage-iphonesimulator/lcov/index.html does not exist.sdkTest
Please help me to resolve this.
Thanks in advance
Kirti
It looks like you were missing something in the tool chain. Did you get this resolved?
Hi Jon,
I was really inspired by your talk at iOSDevUK and started adding tests to my Toolkit Project. To see how good my test coverage was, I also added your Library.
It seems, that it cannot deal with spaces in the folder name, is that possible? Without spaces in the folder name it works like a charm.
Thanks & greetings from Germany,
Jasmin
Hi Jasmin,
I’m pretty sure XcodeCoverage was fixed to handle white spaces. I wonder if something else in the tool chain has broken.
By the way, this is why I avoid spaces in development folder names! This is a common problem across all tools. Unfortunately, it also means the problem can sneak back, because I’m avoiding it in the first place, so how will I know?
Hi Jon
It seems to be causing an error here
DIR=”$( cd “$( dirname “${BASH_SOURCE[0]}” )” && pwd )”
source ${DIR}/envcov.sh
Can you suggest where would you make this fix?
This is so amazing! Thanks a bunch :)
Hi, Jon.
Thank you for your work.
Is it possible to send results by email?
how to do that? Because, mainly, all this stuff are for managers))
or may be upload to Jenkins?
Thanks again.
Hi Jon
Great tool! Though I’m getting some incorrect coverage of classes that are definitely not tested. It’s some UI related classes loaded in the beginning of the app. Since the app is launched when testing I have a suspicion that these classes gets “hits” from this launch? Could this be the case?
Yes! Try applying https://qualitycoding.org/app-delegate-for-tests/ to use a different app delegate during testing, one that doesn’t start a bunch of stuff.
Xcode 7.2 ? I am using KIF for unit test cases automation .
Reading tracefile Coverage.info
lcov: ERROR: no valid records found in tracefile Coverage.info
Reading data file Coverage.info
genhtml: ERROR: no valid records found in tracefile Coverage.info
No data in Coverage.info file . because of that html page is not creating.
Sorry Manoj, I have no idea. I’ve certainly used coverage with Xcode 7.2, though.
would like to know how to find out what classes were touched by a test. Is there a way to do that?
Clear out coverage stats, then run a single test. That should show you the code that was hit by that test.
…I’m curious, how would you use this information?