Code Coverage Fixed for Xcode 5.1

March 30, 2014 — 26 Comments

XcodeCoverage - code coverage for Xcode

The Xcode 5.1 update proved to be more exciting than I anticipated. And I mean “exciting” in the sense of “breaking the tools I use.” xctool stopped working, and so did my code coverage scripts.

(Lesson learned: Always download the developer preview and give it a try, before running Software Update…)

Facebook updated xctool. And I’m happy to say that XcodeCoverage now works with Xcode 5.1! Thanks to my coworker Mike Maietta for figuring out how to create a wrapper around the new gcov so that it continues to work with lcov.

Your coverage numbers may change slightly as you use Xcode 5.1 — but not much. And we don’t need the __gcov_flush() workaround anymore. Hurrah!

Bonus: “Generate code coverage report?” prompt

Finally, I incorporated a submission by Matthew Purland which gives you a way to have XcodeCoverage ask whether you want code coverage whenever you run unit tests. If you want to give a try, edit your Xcode scheme and spin down Test to show Post-actions. Add a New Run Script Action, then do the following:

  • Set “Shell” to /bin/bash
  • Set “Provide build settings from” to your main target (in the example below, I’m doing this for OCHamcrest)
  • Set the script to source ${SRCROOT}/XcodeCoverage/run_code_coverage_post.sh

Post-actions to get code coverage after unit tests

Once you do that, run your unit tests. Once they complete, you should see this prompt:

Generate code coverage report?

Click OK, and XcodeCoverage will run for a while, then show you the coverage results. Pretty slick. (Of course, don’t add this optional step to any shared schemes used in automation.)

…Again, go get the latest XcodeCoverage to get up & running again on Xcode 5.1. Enjoy!

Hole photo by Bart Everson (license)

Jon Reid

Posts Twitter Facebook Google+

I'm passionate about not just improving our code, but improving the way we code.

26 responses to Code Coverage Fixed for Xcode 5.1

  1. Jason J. W. Williams March 31, 2014 at 9:16 pm

    After updating according to the instructions above, my Test target still runs coverage on every invocation, I don’t get a prompt for coverage or not.

    • It could be due to the space in your folder name (which you mention in another response). I had the same problem until I removed the space in my folder name. FWIW I tried to rework the scripts so they’ve be more space tolerant, but just dug myself into more of a hole (bash is not my strong suit)

    • Any luck on this? BTW there have been a few patches to XcodeCoverage recently.

  2. Hey!
    Your tool is awesome, thank you a lot.

    Do you know if coverage files generated by Xcode can support branch coverage? Tried adding key
    –rc lcov_branch_coverage=1
    to lcov calls and didn’t get anything, assuming that coverage files don’t provide that info.

  3. Hi,

    I have followed the steps you have mentioned above for Xcode 5.1 but am getting the following:

    /x86_64/iOS_MobileSDK.gcda!
    Finished .info-file creation
    Reading tracefile Coverage.info
    lcov: ERROR: no valid records found in tracefile Coverage.info
    Reading tracefile Coverage.info
    lcov: ERROR: no valid records found in tracefile Coverage.info
    Reading data file Coverage.info
    genhtml: ERROR: no valid records found in tracefile Coverage.info

    thanks,

    Sana

  4. Love the tool, thanks a lot.

    I see gcda and gcno files in two different directories. One directory is named appName.build/ and the other is appNameTests.build/ (basically each target has its own directory). There are gcda/gcno files for every file in the project, duplicated in each directory. The only thing is that the coverage metrics captured in those files are vastly different. So my question is, which metrics are to be believed? What is the difference in those files?

    Just trying to raise some visibility to this question. I also posted it as an issue on XcodeCoverage in github, and here’s a reference to the same question on stackoverflow

    http://stackoverflow.com/questions/22188093/app-builds-gcda-gcno-files-vs-apptests-builds-gcda-gcno-files

    Thanks, and keep testing

  5. Another question – I often follow the pattern where I use OCMock’s partial mock objects, to partially mock the class under test (the first section on this blog shows a write up of the pattern https://engineering.aweber.com/improving-ios-unit-tests-with-ocmock/).

    The problem though, is that using XcodeCoverage, it shows 0 lines covered on any non-mocked method invoked on a partial mock. Obviously I know that not to be the case. Any ideas what’s up? CoverStory shows the same issue.

    I’m loading OCMock via a cocoapod if that makes a difference.

  6. I can generate code coverage but only partially. I always get segmentation fault. :(

    XcodeCoverage/llvm-cov-wrapper.sh: line 13: 15665 Segmentation fault: 11  /usr/bin/gcov $*

    Does anyone have the same issue?

  7. Thanks for keeping things up to date – I love this particular tool and use it regularly.

    One nit – it may be true that the _gov_flush() workaround isn’t required if your project is based on Apple’s newer XCTest framework, but if your test code is still based on the older OCUnit framework, it does still appear to be necessary.

  8. Jason J. W. Williams April 8, 2014 at 1:41 pm

    Also, it still doesn’t handle paths with spaces properly.

  9. Jason J. W. Williams April 9, 2014 at 11:12 am

    Does anyone else see a ton of these statements in your build output when generating coverage data with your tests:

    profiling:invalid arc tag (0x726f7461)
    profiling:invalid arc tag (0x00af57e9)
    profiling:invalid arc tag (0×00000012)
    profiling:invalid number of counters (10)

    • Looks like we are running across the same problems. If you get any results, please add them as comments here.

    • I was getting this too, then I did a clean and rebuild. The message has gone away. I think it could have been related to calls to __gcov_flush, which I’ve now removed. Apparently they are not needed if you’re using XCTest.

  10. Fernando Canon April 15, 2014 at 1:16 pm

    Since upgrade to xcode 5.1.1 Im getting a crash on OCMockito, when calling methods from a mock object. This is the console message:

    [NSInvocation mkt_arrayArguments]: unrecognized selector sent to instance 0x2864db0
    :0: error: -[ExampleTests testUsingNumbers] : -[NSInvocation mkt_arrayArguments]: unrecognized selector sent to instance 0x2864db0
    (
        0   CoreFoundation                      0x00b1d5e4 __exceptionPreprocess + 180
        1   libobjc.A.dylib                     0x007958b6 objc_exception_throw + 44
        2   CoreFoundation                      0x00bba903 -[NSObject(NSObject) doesNotRecognizeSelector:] + 275
        3   CoreFoundation                      0x00b0d90b ___forwarding___ + 1019
        4   CoreFoundation                      0x00b0d4ee _CF_forwarding_prep_0 + 14
        5   ExampleTests                        0x0279107a -[MKTInvocationMatcher setExpectedInvocation:] + 303
        6   ExampleTests                        0x027901f9 -[MKTInvocationContainer setInvocationForPotentialStubbing:] + 150
        7   ExampleTests                        0x0279251d -[MKTBaseMockObject prepareInvocationForStubbing:] + 52
        8   ExampleTests                        0x027921b9 -[MKTBaseMockObject forwardInvocation:] + 79
        9   CoreFoundation                      0x00b0d6da ___forwarding___ + 458
        10  CoreFoundation                      0x00b0d4ee _CF_forwarding_prep_0 + 14
        11  ExampleTests                        0x0278c7c3 -[ExampleTests testUsingNumbers] + 147
        12  CoreFoundation                      0x00b11d1d __invoking___ + 29
        13  CoreFoundation                      0x00b11c2a -[NSInvocation invoke] + 362
        14  XCTest                              0x201032bf -[XCTestCase invokeTest] + 212
        15  XCTest                              0x2010338d -[XCTestCase performTest:] + 111
        16  XCTest                              0x2010417c -[XCTest run] + 82
        17  XCTest                              0x20102a44 -[XCTestSuite performTest:] + 139
        18  XCTest                              0x2010417c -[XCTest run] + 82
        19  XCTest                              0x20102a44 -[XCTestSuite performTest:] + 139
        20  XCTest                              0x2010417c -[XCTest run] + 82
        21  XCTest                              0x20102a44 -[XCTestSuite performTest:] + 139
        22  XCTest                              0x2010417c -[XCTest run] + 82
        23  XCTest                              0x20105aa1 +[XCTestProbe runTests:] + 183
        24  libobjc.A.dylib                     0x007a7737 +[NSObject performSelector:withObject:] + 70
        25  xctest                              0x0000233e xctest + 4926
        26  xctest                              0x00002590 xctest + 5520
        27  xctest                              0x00002671 xctest + 5745
        28  xctest                              0x00002007 xctest + 4103
        29  libdyld.dylib                       0x01655701 start + 1
    )

    I´ve been struggling with it for a couple of days .. change back xcode version, install the framework from cocoa pods, install previos versions, and so on. But nothing
    Any Ideas??

  11. Am I supposed to implement these changes ONTOP of the changes that you outline here: http://qualitycoding.org/xcode51-code-coverage/#more-1538?

    The problem that I’m running into is that nothing seems to happen after I click the okay button to generate the code report.

    Not getting any linker errors or anything like. that. Kinda confused at what I missed. Everything is installed into the right directory, i’ve installed all the command line tools, etc.

    Thanks in advance to any insight you might have!

    -Craig

  12. First, thanks for providing this, looks quite useful as I’ve seen some reports from it before. Am now trying to get code coverage working for a library target following the steps that were outlined. In Xcode 5.1 it says:

    . . ./Build/Products/Debug-iphonesimulator ~/workspace/sdk/ios/x/XcodeCoverage ~/workspace/sdk/ios/x/XcodeCoverage
    Capturing coverage data from . . ./Build/Intermediates/x.build/Debug-iphonesimulator/x.build/Objects-normal/x86_64
    Found gcov version: 4.2.1
    Scanning . . ./Build/Intermediates/x.build/Debug-iphonesimulator/x.build/Objects-normal/x86_64 for .gcda files …
    geninfo: ERROR: no .gcda files found in . . ./Build/Intermediates/x.build/Debug-iphonesimulator/x.build/Objects-normal/x86_64!
    Reading tracefile Coverage.info
    lcov: ERROR: no valid records found in tracefile Coverage.info
    Reading tracefile Coverage.info
    lcov: ERROR: no valid records found in tracefile Coverage.info
    Reading data file Coverage.info
    genhtml: ERROR: no valid records found in tracefile Coverage.info

    The .gcno files are there but the .gcda files are not. For the i386 build, there are both .gcno and .gcda files.

    Steps:
    Instrument Program Flow Yes for debug
    Generate Test Coverage Files Yes for debug
    Added XcodeCoverage/exportenv.sh to Run Script of library target
    Run unit tests (kiwi) with Cmd+U. Unit tests are a separate target but rely on the library target I’m trying to get coverage on.

    I tried adding:
    -lgcov to Other Linker Flags
    Set “Application does not run in background” to Yes in the Unit tests target

    Not sure what else to try. Anyone have any advice?

  13. I wonder if code coverage can (and how?) include Development Pods folder. For now when I’m using standard way (described here) I get the report which doesn’t include *.m files from Pod (just *.h file).
    The second problem is to find a way to have code coverage only for methods called by written Unit Tests, and not all methods called by system during testing.

  14. Seems that the “env.sh” file was missing?

  15. Apple has their own ‘how-to’ they released in March 2014 (according to Google’s search results) that explains how to hook GCOV.

    https://developer.apple.com/library/ios/qa/qa1514/_index.html

Leave a Reply

*

Text formatting is available via select HTML.

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> 

Have you Subscribed yet? Don't miss a post!