The Xcode 5.1 update proved to be more exciting than I anticipated. And I mean “exciting” in the sense of “breaking the tools I use.” xctool stopped working, and so did my code coverage scripts.
(Lesson learned: Always download the developer preview and give it a try, before running Software Update…)
Facebook updated xctool. And I’m happy to say that XcodeCoverage now works with Xcode 5.1! Thanks to my coworker Mike Maietta for figuring out how to create a wrapper around the new gcov so that it continues to work with lcov.
Your coverage numbers may change slightly as you use Xcode 5.1—but not much. And we don’t need the __gcov_flush() workaround anymore. Hurrah!
Bonus: “Generate code coverage report?” prompt
Finally, I incorporated a submission by Matthew Purland which gives you a way to have XcodeCoverage ask whether you want code coverage whenever you run unit tests. If you want to give a try, edit your Xcode scheme and spin down Test to show Post-actions. Add a New Run Script Action, then do the following:
- Set “Shell” to
/bin/bash - Set “Provide build settings from” to your main target (in the example below, I’m doing this for OCHamcrest)
- Set the script to
source ${SRCROOT}/XcodeCoverage/run_code_coverage_post.sh
Once you do that, run your unit tests. Once they complete, you should see this prompt:
Click OK, and XcodeCoverage will run for a while, then show you the coverage results. Pretty slick. (Of course, don’t add this optional step to any shared schemes used in automation.)
…Again, go get the latest XcodeCoverage to get up & running again on Xcode 5.1. Enjoy!
After updating according to the instructions above, my Test target still runs coverage on every invocation, I don’t get a prompt for coverage or not.
It could be due to the space in your folder name (which you mention in another response). I had the same problem until I removed the space in my folder name. FWIW I tried to rework the scripts so they’ve be more space tolerant, but just dug myself into more of a hole (bash is not my strong suit)
Any luck on this? BTW there have been a few patches to XcodeCoverage recently.
Hey!
Your tool is awesome, thank you a lot.
Do you know if coverage files generated by Xcode can support branch coverage? Tried adding key
–rc lcov_branch_coverage=1
to lcov calls and didn’t get anything, assuming that coverage files don’t provide that info.
I assume you’re right; we’re stuck with whatever Apple decides.
Hi,
I have followed the steps you have mentioned above for Xcode 5.1 but am getting the following:
/x86_64/iOS_MobileSDK.gcda!
Finished .info-file creation
Reading tracefile Coverage.info
lcov: ERROR: no valid records found in tracefile Coverage.info
Reading tracefile Coverage.info
lcov: ERROR: no valid records found in tracefile Coverage.info
Reading data file Coverage.info
genhtml: ERROR: no valid records found in tracefile Coverage.info
thanks,
Sana
Sana, did you manage to solve this?
Love the tool, thanks a lot.
I see gcda and gcno files in two different directories. One directory is named appName.build/ and the other is appNameTests.build/ (basically each target has its own directory). There are gcda/gcno files for every file in the project, duplicated in each directory. The only thing is that the coverage metrics captured in those files are vastly different. So my question is, which metrics are to be believed? What is the difference in those files?
Just trying to raise some visibility to this question. I also posted it as an issue on XcodeCoverage in github, and here’s a reference to the same question on stackoverflow
http://stackoverflow.com/questions/22188093/app-builds-gcda-gcno-files-vs-apptests-builds-gcda-gcno-files
Thanks, and keep testing
Another question – I often follow the pattern where I use OCMock’s partial mock objects, to partially mock the class under test (the first section on this blog shows a write up of the pattern https://engineering.aweber.com/improving-ios-unit-tests-with-ocmock/).
The problem though, is that using XcodeCoverage, it shows 0 lines covered on any non-mocked method invoked on a partial mock. Obviously I know that not to be the case. Any ideas what’s up? CoverStory shows the same issue.
I’m loading OCMock via a cocoapod if that makes a difference.
I wonder if this is a side effect of the way OCMock does partial mocking. Try reaching out to Erik Dörnenburg.
I can generate code coverage but only partially. I always get segmentation fault. :(
XcodeCoverage/llvm-cov-wrapper.sh: line 13: 15665 Segmentation fault: 11 /usr/bin/gcov $*Does anyone have the same issue?
Thanks for keeping things up to date – I love this particular tool and use it regularly.
One nit – it may be true that the _gov_flush() workaround isn’t required if your project is based on Apple’s newer XCTest framework, but if your test code is still based on the older OCUnit framework, it does still appear to be necessary.
Yes, it seems Apple got the message that the workaround was a pain.
Also, it still doesn’t handle paths with spaces properly.
I’m running into this as well. My project has scheme, target and configuration names with spaces in them. Is there a workaround?
It does now.
Does anyone else see a ton of these statements in your build output when generating coverage data with your tests:
profiling:invalid arc tag (0x726f7461)
profiling:invalid arc tag (0x00af57e9)
profiling:invalid arc tag (0x00000012)
profiling:invalid number of counters (10)
Looks like we are running across the same problems. If you get any results, please add them as comments here.
I was getting this too, then I did a clean and rebuild. The message has gone away. I think it could have been related to calls to __gcov_flush, which I’ve now removed. Apparently they are not needed if you’re using XCTest.
Since upgrade to xcode 5.1.1 Im getting a crash on OCMockito, when calling methods from a mock object. This is the console message:
[NSInvocation mkt_arrayArguments]: unrecognized selector sent to instance 0x2864db0
:0: error: -[ExampleTests testUsingNumbers] : -[NSInvocation mkt_arrayArguments]: unrecognized selector sent to instance 0x2864db0
(
0 CoreFoundation 0x00b1d5e4 __exceptionPreprocess + 180
1 libobjc.A.dylib 0x007958b6 objc_exception_throw + 44
2 CoreFoundation 0x00bba903 -[NSObject(NSObject) doesNotRecognizeSelector:] + 275
3 CoreFoundation 0x00b0d90b ___forwarding___ + 1019
4 CoreFoundation 0x00b0d4ee _CF_forwarding_prep_0 + 14
5 ExampleTests 0x0279107a -[MKTInvocationMatcher setExpectedInvocation:] + 303
6 ExampleTests 0x027901f9 -[MKTInvocationContainer setInvocationForPotentialStubbing:] + 150
7 ExampleTests 0x0279251d -[MKTBaseMockObject prepareInvocationForStubbing:] + 52
8 ExampleTests 0x027921b9 -[MKTBaseMockObject forwardInvocation:] + 79
9 CoreFoundation 0x00b0d6da ___forwarding___ + 458
10 CoreFoundation 0x00b0d4ee _CF_forwarding_prep_0 + 14
11 ExampleTests 0x0278c7c3 -[ExampleTests testUsingNumbers] + 147
12 CoreFoundation 0x00b11d1d __invoking___ + 29
13 CoreFoundation 0x00b11c2a -[NSInvocation invoke] + 362
14 XCTest 0x201032bf -[XCTestCase invokeTest] + 212
15 XCTest 0x2010338d -[XCTestCase performTest:] + 111
16 XCTest 0x2010417c -[XCTest run] + 82
17 XCTest 0x20102a44 -[XCTestSuite performTest:] + 139
18 XCTest 0x2010417c -[XCTest run] + 82
19 XCTest 0x20102a44 -[XCTestSuite performTest:] + 139
20 XCTest 0x2010417c -[XCTest run] + 82
21 XCTest 0x20102a44 -[XCTestSuite performTest:] + 139
22 XCTest 0x2010417c -[XCTest run] + 82
23 XCTest 0x20105aa1 +[XCTestProbe runTests:] + 183
24 libobjc.A.dylib 0x007a7737 +[NSObject performSelector:withObject:] + 70
25 xctest 0x0000233e xctest + 4926
26 xctest 0x00002590 xctest + 5520
27 xctest 0x00002671 xctest + 5745
28 xctest 0x00002007 xctest + 4103
29 libdyld.dylib 0x01655701 start + 1
)
I´ve been struggling with it for a couple of days .. change back xcode version, install the framework from cocoa pods, install previos versions, and so on. But nothing
Any Ideas??
Make sure you’re using the latest release of OCMockito, together with the latest release of OCHamcrest.
Am I supposed to implement these changes ONTOP of the changes that you outline here: https://qualitycoding.org/xcode51-code-coverage/#more-1538?
The problem that I’m running into is that nothing seems to happen after I click the okay button to generate the code report.
Not getting any linker errors or anything like. that. Kinda confused at what I missed. Everything is installed into the right directory, i’ve installed all the command line tools, etc.
Thanks in advance to any insight you might have!
-Craig
Craig, the __gcov_flush() workaround is no longer necessary. Are you the .gcda files getting generated?
since upgrade to xcode 5.1, when I analyze the coverage data, I got an error like this:
gcov(25639,0x7fff72849310) malloc: *** error for object 0x101002d40: pointer being realloc’d was not allocated
*** set a breakpoint in malloc_error_break to debug
Stack dump:
0. Program arguments: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/gcov /Users/ted/Desktop/iOSCodeDebug/Coverage/Coverage_armv7s/CoverageTest_armv7s/20140527155010/SongDBAdapter.gcda -o /Users/ted/Desktop/iOSCodeDebug/Coverage/Coverage_armv7s/CoverageTest_armv7s/20140527155010
/Users/ted/Desktop/iOSCodeDebug/lcov/XcodeCoverage/llvm-cov-wrapper.sh: line 13: 25639 Abort trap: 6 /usr/bin/gcov $*
I don’t know why this error occured，do you have any ideas?
Ted, did you ever resolve this?
First, thanks for providing this, looks quite useful as I’ve seen some reports from it before. Am now trying to get code coverage working for a library target following the steps that were outlined. In Xcode 5.1 it says:
. . ./Build/Products/Debug-iphonesimulator ~/workspace/sdk/ios/x/XcodeCoverage ~/workspace/sdk/ios/x/XcodeCoverage
Capturing coverage data from . . ./Build/Intermediates/x.build/Debug-iphonesimulator/x.build/Objects-normal/x86_64
Found gcov version: 4.2.1
Scanning . . ./Build/Intermediates/x.build/Debug-iphonesimulator/x.build/Objects-normal/x86_64 for .gcda files …
geninfo: ERROR: no .gcda files found in . . ./Build/Intermediates/x.build/Debug-iphonesimulator/x.build/Objects-normal/x86_64!
Reading tracefile Coverage.info
lcov: ERROR: no valid records found in tracefile Coverage.info
Reading tracefile Coverage.info
lcov: ERROR: no valid records found in tracefile Coverage.info
Reading data file Coverage.info
genhtml: ERROR: no valid records found in tracefile Coverage.info
The .gcno files are there but the .gcda files are not. For the i386 build, there are both .gcno and .gcda files.
Steps:
Instrument Program Flow Yes for debug
Generate Test Coverage Files Yes for debug
Added XcodeCoverage/exportenv.sh to Run Script of library target
Run unit tests (kiwi) with Cmd+U. Unit tests are a separate target but rely on the library target I’m trying to get coverage on.
I tried adding:
-lgcov to Other Linker Flags
Set “Application does not run in background” to Yes in the Unit tests target
Not sure what else to try. Anyone have any advice?
Ryan, are these problems behind you with a updated versions of Xcode and lcov?
I wonder if code coverage can (and how?) include Development Pods folder. For now when I’m using standard way (described here) I get the report which doesn’t include *.m files from Pod (just *.h file).
The second problem is to find a way to have code coverage only for methods called by written Unit Tests, and not all methods called by system during testing.
Konrad,
Yes, you can add coverage for pods and other nested projects.
First, you have to track down the object directory directory for the project you want to add.
Then edit envcov.sh, adding a definition for the new directory. For example:
# Added shared lib
LIB_OBJ=${OBJROOT}/ANJShared.build/Debug-iphonesimulator/ANJShared.build/Objects-normal/${NATIVE_ARCH}
Finally, edit getcov and add some lines to gather_coverage(). What you want to do is capture the data to a different tracefile, then add all the tracefiles together. For example:
gather_coverage()
{
LCOV --capture --derive-func-data -b "${SRCROOT}" -d "${OBJ_DIR}" -o ${LCOV_INFO}
# Added shared lib
LCOV --capture --derive-func-data -b "${SRCROOT}" -d "${LIB_OBJ}" -o Shared.info
LCOV -a ${LCOV_INFO} -a Shared.info -o ${LCOV_INFO}
}
Seems that the “env.sh” file was missing?
env.sh is a generated file specific to your setup. Per the instructions: Add a Run Script build phase to execute XcodeCoverage/exportenv.sh
Apple has their own ‘how-to’ they released in March 2014 (according to Google’s search results) that explains how to hook GCOV.
https://developer.apple.com/library/ios/qa/qa1514/_index.html
Thanks, Adam! Some of that information is now dated, while other parts are new to me.
Jon, thanks for sharing this. Our team has been working on Xcode 6.1 with IOS 8 sdk, supporting both 32-bit and 64-bit. According to this article on apple’s website(https://developer.apple.com/library/ios/qa/qa1514/_index.html), I just set the UIApplicationExitsOnSuspend key to YES to force coverage data to be written out to .gcda files on mobile. But we have more than 2600 .gcno files in our application, so there are always some .gcda file cannot be written out in the short time. Another issue is that the output .ipa with coverage information cannot be installed on some 32-bit iphone. Any idea how to solve this? thank you very much!
Maggie, that document is dated; there’s no longer any need to force the data to be flushed.
Are you trying to get coverage information on real devices instead of the simulator? If so, what is actually driving the app that you want to measure?
Hi! thanks for that tool, it looks very promising for my work. I’ve been using gcov following apple’s method linked by Adam, and visualizing the reports with CoverStory. But I would like to be able to generate html reports for portability reasons. Unfortunately, I can’t use lcov. I get the following error every time I try to use geninfo:
geninfo: ERROR: myfile.gcno: found unrecognized record format
I’m running OS X 10.9.5 with all the last updates including Xcode 6 and the last version of the Command Line Tools. Any idea how to solve that?
Thanks a lot!
I’ve been encountered the same issue with you. There are more than 2600 .gcno files in our application, and it generates the error (geninfo: ERROR: myfile.gcno: found unrecognized record format) for about 100 .gcno files. Did you ever resolve this?
Hi Jon,
I’m in the process of separating out parts of quite a large app into separate libraries which will be loaded via CocoaPods. I’ve set up these libraries using the pod lib create command, which creates a library from a template, with an Example project set up all ready to use the library. Tests you write are actually written against the example project. With this setup I’m struggling to figure out how to set up XcodeCoverage. If I do it normally, I end up just showing the coverage of the files in the Example project, instead of the coverage of the actual library that I’m writing. Do you have any insight into how I could alter the exportenv.sh file to properly show coverage of my library instead of the Example project?
Sincerely,
Ben.
I am looking for this as well. My library is on development pods and I was wondering how to get the test coverage for the library instead of the example project.