Code Coverage Fixed for Xcode 5.1

March 30, 2014 — 35 Comments

The Xcode 5.1 update proved to be more exciting than I anticipated. And I mean “exciting” in the sense of “breaking the tools I use.” xctool stopped working, and so did my code coverage scripts.

XcodeCoverage - code coverage for Xcode

Hole” by Bart Everson, used under CC BY 2.0 / Cropped

(Lesson learned: Always download the developer preview and give it a try, before running Software Update…)

Facebook updated xctool. And I’m happy to say that XcodeCoverage now works with Xcode 5.1! Thanks to my coworker Mike Maietta for figuring out how to create a wrapper around the new gcov so that it continues to work with lcov.

Your coverage numbers may change slightly as you use Xcode 5.1 — but not much. And we don’t need the __gcov_flush() workaround anymore. Hurrah!

Bonus: “Generate code coverage report?” prompt

Finally, I incorporated a submission by Matthew Purland which gives you a way to have XcodeCoverage ask whether you want code coverage whenever you run unit tests. If you want to give a try, edit your Xcode scheme and spin down Test to show Post-actions. Add a New Run Script Action, then do the following:

  • Set “Shell” to /bin/bash
  • Set “Provide build settings from” to your main target (in the example below, I’m doing this for OCHamcrest)
  • Set the script to source ${SRCROOT}/XcodeCoverage/run_code_coverage_post.sh

Post-actions to get code coverage after unit tests

Once you do that, run your unit tests. Once they complete, you should see this prompt:

Generate code coverage report?

Click OK, and XcodeCoverage will run for a while, then show you the coverage results. Pretty slick. (Of course, don’t add this optional step to any shared schemes used in automation.)

…Again, go get the latest XcodeCoverage to get up & running again on Xcode 5.1. Enjoy!

Disclosure of Material Connection: Some of the links in the post above are "affiliate links." This means if you click on the link and purchase the item, I will receive an affiliate commission. Regardless, I only recommend products or services I use personally and believe will add value to my readers. I am disclosing this in accordance with the Federal Trade Commission’s 16 CFR, Part 255: "Guides Concerning the Use of Endorsements and Testimonials in Advertising."

Enjoyed this article? Sign up to get future articles by email.

Email Address:

35 responses to Code Coverage Fixed for Xcode 5.1

  1. Jason J. W. Williams March 31, 2014 at 9:16 pm

    After updating according to the instructions above, my Test target still runs coverage on every invocation, I don’t get a prompt for coverage or not.

    • It could be due to the space in your folder name (which you mention in another response). I had the same problem until I removed the space in my folder name. FWIW I tried to rework the scripts so they’ve be more space tolerant, but just dug myself into more of a hole (bash is not my strong suit)

    • Any luck on this? BTW there have been a few patches to XcodeCoverage recently.

  2. Hey!
    Your tool is awesome, thank you a lot.

    Do you know if coverage files generated by Xcode can support branch coverage? Tried adding key
    –rc lcov_branch_coverage=1
    to lcov calls and didn’t get anything, assuming that coverage files don’t provide that info.

  3. Hi,

    I have followed the steps you have mentioned above for Xcode 5.1 but am getting the following:

    /x86_64/iOS_MobileSDK.gcda!
    Finished .info-file creation
    Reading tracefile Coverage.info
    lcov: ERROR: no valid records found in tracefile Coverage.info
    Reading tracefile Coverage.info
    lcov: ERROR: no valid records found in tracefile Coverage.info
    Reading data file Coverage.info
    genhtml: ERROR: no valid records found in tracefile Coverage.info

    thanks,

    Sana

  4. Love the tool, thanks a lot.

    I see gcda and gcno files in two different directories. One directory is named appName.build/ and the other is appNameTests.build/ (basically each target has its own directory). There are gcda/gcno files for every file in the project, duplicated in each directory. The only thing is that the coverage metrics captured in those files are vastly different. So my question is, which metrics are to be believed? What is the difference in those files?

    Just trying to raise some visibility to this question. I also posted it as an issue on XcodeCoverage in github, and here’s a reference to the same question on stackoverflow

    http://stackoverflow.com/questions/22188093/app-builds-gcda-gcno-files-vs-apptests-builds-gcda-gcno-files

    Thanks, and keep testing

  5. Another question – I often follow the pattern where I use OCMock’s partial mock objects, to partially mock the class under test (the first section on this blog shows a write up of the pattern https://engineering.aweber.com/improving-ios-unit-tests-with-ocmock/).

    The problem though, is that using XcodeCoverage, it shows 0 lines covered on any non-mocked method invoked on a partial mock. Obviously I know that not to be the case. Any ideas what’s up? CoverStory shows the same issue.

    I’m loading OCMock via a cocoapod if that makes a difference.

  6. I can generate code coverage but only partially. I always get segmentation fault. :(

    XcodeCoverage/llvm-cov-wrapper.sh: line 13: 15665 Segmentation fault: 11  /usr/bin/gcov $*

    Does anyone have the same issue?

  7. Thanks for keeping things up to date – I love this particular tool and use it regularly.

    One nit – it may be true that the _gov_flush() workaround isn’t required if your project is based on Apple’s newer XCTest framework, but if your test code is still based on the older OCUnit framework, it does still appear to be necessary.

  8. Jason J. W. Williams April 8, 2014 at 1:41 pm

    Also, it still doesn’t handle paths with spaces properly.

  9. Jason J. W. Williams April 9, 2014 at 11:12 am

    Does anyone else see a ton of these statements in your build output when generating coverage data with your tests:

    profiling:invalid arc tag (0x726f7461)
    profiling:invalid arc tag (0x00af57e9)
    profiling:invalid arc tag (0x00000012)
    profiling:invalid number of counters (10)

    • Looks like we are running across the same problems. If you get any results, please add them as comments here.

    • I was getting this too, then I did a clean and rebuild. The message has gone away. I think it could have been related to calls to __gcov_flush, which I’ve now removed. Apparently they are not needed if you’re using XCTest.

  10. Fernando Canon April 15, 2014 at 1:16 pm

    Since upgrade to xcode 5.1.1 Im getting a crash on OCMockito, when calling methods from a mock object. This is the console message:

    [NSInvocation mkt_arrayArguments]: unrecognized selector sent to instance 0x2864db0
    :0: error: -[ExampleTests testUsingNumbers] : -[NSInvocation mkt_arrayArguments]: unrecognized selector sent to instance 0x2864db0
    (
        0   CoreFoundation                      0x00b1d5e4 __exceptionPreprocess + 180
        1   libobjc.A.dylib                     0x007958b6 objc_exception_throw + 44
        2   CoreFoundation                      0x00bba903 -[NSObject(NSObject) doesNotRecognizeSelector:] + 275
        3   CoreFoundation                      0x00b0d90b ___forwarding___ + 1019
        4   CoreFoundation                      0x00b0d4ee _CF_forwarding_prep_0 + 14
        5   ExampleTests                        0x0279107a -[MKTInvocationMatcher setExpectedInvocation:] + 303
        6   ExampleTests                        0x027901f9 -[MKTInvocationContainer setInvocationForPotentialStubbing:] + 150
        7   ExampleTests                        0x0279251d -[MKTBaseMockObject prepareInvocationForStubbing:] + 52
        8   ExampleTests                        0x027921b9 -[MKTBaseMockObject forwardInvocation:] + 79
        9   CoreFoundation                      0x00b0d6da ___forwarding___ + 458
        10  CoreFoundation                      0x00b0d4ee _CF_forwarding_prep_0 + 14
        11  ExampleTests                        0x0278c7c3 -[ExampleTests testUsingNumbers] + 147
        12  CoreFoundation                      0x00b11d1d __invoking___ + 29
        13  CoreFoundation                      0x00b11c2a -[NSInvocation invoke] + 362
        14  XCTest                              0x201032bf -[XCTestCase invokeTest] + 212
        15  XCTest                              0x2010338d -[XCTestCase performTest:] + 111
        16  XCTest                              0x2010417c -[XCTest run] + 82
        17  XCTest                              0x20102a44 -[XCTestSuite performTest:] + 139
        18  XCTest                              0x2010417c -[XCTest run] + 82
        19  XCTest                              0x20102a44 -[XCTestSuite performTest:] + 139
        20  XCTest                              0x2010417c -[XCTest run] + 82
        21  XCTest                              0x20102a44 -[XCTestSuite performTest:] + 139
        22  XCTest                              0x2010417c -[XCTest run] + 82
        23  XCTest                              0x20105aa1 +[XCTestProbe runTests:] + 183
        24  libobjc.A.dylib                     0x007a7737 +[NSObject performSelector:withObject:] + 70
        25  xctest                              0x0000233e xctest + 4926
        26  xctest                              0x00002590 xctest + 5520
        27  xctest                              0x00002671 xctest + 5745
        28  xctest                              0x00002007 xctest + 4103
        29  libdyld.dylib                       0x01655701 start + 1
    )

    I´ve been struggling with it for a couple of days .. change back xcode version, install the framework from cocoa pods, install previos versions, and so on. But nothing
    Any Ideas??

  11. Am I supposed to implement these changes ONTOP of the changes that you outline here: http://qualitycoding.org/xcode51-code-coverage/#more-1538?

    The problem that I’m running into is that nothing seems to happen after I click the okay button to generate the code report.

    Not getting any linker errors or anything like. that. Kinda confused at what I missed. Everything is installed into the right directory, i’ve installed all the command line tools, etc.

    Thanks in advance to any insight you might have!

    -Craig

  12. since upgrade to xcode 5.1, when I analyze the coverage data, I got an error like this:
    gcov(25639,0x7fff72849310) malloc: *** error for object 0x101002d40: pointer being realloc’d was not allocated
    *** set a breakpoint in malloc_error_break to debug
    Stack dump:
    0. Program arguments: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/gcov /Users/ted/Desktop/iOSCodeDebug/Coverage/Coverage_armv7s/CoverageTest_armv7s/20140527155010/SongDBAdapter.gcda -o /Users/ted/Desktop/iOSCodeDebug/Coverage/Coverage_armv7s/CoverageTest_armv7s/20140527155010
    /Users/ted/Desktop/iOSCodeDebug/lcov/XcodeCoverage/llvm-cov-wrapper.sh: line 13: 25639 Abort trap: 6 /usr/bin/gcov $*

    I don’t know why this error occured,do you have any ideas?

  13. First, thanks for providing this, looks quite useful as I’ve seen some reports from it before. Am now trying to get code coverage working for a library target following the steps that were outlined. In Xcode 5.1 it says:

    . . ./Build/Products/Debug-iphonesimulator ~/workspace/sdk/ios/x/XcodeCoverage ~/workspace/sdk/ios/x/XcodeCoverage
    Capturing coverage data from . . ./Build/Intermediates/x.build/Debug-iphonesimulator/x.build/Objects-normal/x86_64
    Found gcov version: 4.2.1
    Scanning . . ./Build/Intermediates/x.build/Debug-iphonesimulator/x.build/Objects-normal/x86_64 for .gcda files …
    geninfo: ERROR: no .gcda files found in . . ./Build/Intermediates/x.build/Debug-iphonesimulator/x.build/Objects-normal/x86_64!
    Reading tracefile Coverage.info
    lcov: ERROR: no valid records found in tracefile Coverage.info
    Reading tracefile Coverage.info
    lcov: ERROR: no valid records found in tracefile Coverage.info
    Reading data file Coverage.info
    genhtml: ERROR: no valid records found in tracefile Coverage.info

    The .gcno files are there but the .gcda files are not. For the i386 build, there are both .gcno and .gcda files.

    Steps:
    Instrument Program Flow Yes for debug
    Generate Test Coverage Files Yes for debug
    Added XcodeCoverage/exportenv.sh to Run Script of library target
    Run unit tests (kiwi) with Cmd+U. Unit tests are a separate target but rely on the library target I’m trying to get coverage on.

    I tried adding:
    -lgcov to Other Linker Flags
    Set “Application does not run in background” to Yes in the Unit tests target

    Not sure what else to try. Anyone have any advice?

  14. I wonder if code coverage can (and how?) include Development Pods folder. For now when I’m using standard way (described here) I get the report which doesn’t include *.m files from Pod (just *.h file).
    The second problem is to find a way to have code coverage only for methods called by written Unit Tests, and not all methods called by system during testing.

    • Konrad,

      Yes, you can add coverage for pods and other nested projects.

      First, you have to track down the object directory directory for the project you want to add.

      Then edit envcov.sh, adding a definition for the new directory. For example:

      # Added shared lib
      LIB_OBJ=${OBJROOT}/ANJShared.build/Debug-iphonesimulator/ANJShared.build/Objects-normal/${NATIVE_ARCH}

      Finally, edit getcov and add some lines to gather_coverage(). What you want to do is capture the data to a different tracefile, then add all the tracefiles together. For example:

      gather_coverage()
      {
          LCOV --capture --derive-func-data -b "${SRCROOT}" -d "${OBJ_DIR}" -o ${LCOV_INFO}

          # Added shared lib
          LCOV --capture --derive-func-data -b "${SRCROOT}" -d "${LIB_OBJ}" -o Shared.info
          LCOV -a ${LCOV_INFO} -a Shared.info -o ${LCOV_INFO}
      }
  15. Seems that the “env.sh” file was missing?

  16. Apple has their own ‘how-to’ they released in March 2014 (according to Google’s search results) that explains how to hook GCOV.

    https://developer.apple.com/library/ios/qa/qa1514/_index.html

  17. Vincent Le Bourlot October 9, 2014 at 7:58 am

    Hi! thanks for that tool, it looks very promising for my work. I’ve been using gcov following apple’s method linked by Adam, and visualizing the reports with CoverStory. But I would like to be able to generate html reports for portability reasons. Unfortunately, I can’t use lcov. I get the following error every time I try to use geninfo:

    geninfo: ERROR: myfile.gcno: found unrecognized record format

    I’m running OS X 10.9.5 with all the last updates including Xcode 6 and the last version of the Command Line Tools. Any idea how to solve that?
    Thanks a lot!

Leave a Reply

*

Text formatting is available via select HTML.

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> 

Have you Subscribed yet? Don't miss a post!