Code Coverage Fixed for Xcode 5.1

Shares

The Xcode 5.1 update proved to be more exciting than I anticipated. And I mean “exciting” in the sense of “breaking the tools I use.” xctool stopped working, and so did my code coverage scripts.

XcodeCoverage - code coverage for Xcode

Hole by Bart Everson, used under CC BY 2.0 / Cropped original and added logo and title

(Lesson learned: Always download the developer preview and give it a try, before running Software Update…)

Facebook updated xctool. And I’m happy to say that XcodeCoverage now works with Xcode 5.1! Thanks to my coworker Mike Maietta for figuring out how to create a wrapper around the new gcov so that it continues to work with lcov.

Your coverage numbers may change slightly as you use Xcode 5.1 — but not much. And we don’t need the __gcov_flush() workaround anymore. Hurrah!

Bonus: “Generate code coverage report?” prompt

Finally, I incorporated a submission by Matthew Purland which gives you a way to have XcodeCoverage ask whether you want code coverage whenever you run unit tests. If you want to give a try, edit your Xcode scheme and spin down Test to show Post-actions. Add a New Run Script Action, then do the following:

  • Set “Shell” to /bin/bash
  • Set “Provide build settings from” to your main target (in the example below, I’m doing this for OCHamcrest)
  • Set the script to source ${SRCROOT}/XcodeCoverage/run_code_coverage_post.sh

Post-actions to get code coverage after unit tests

Once you do that, run your unit tests. Once they complete, you should see this prompt:

Generate code coverage report?

Click OK, and XcodeCoverage will run for a while, then show you the coverage results. Pretty slick. (Of course, don’t add this optional step to any shared schemes used in automation.)

…Again, go get the latest XcodeCoverage to get up & running again on Xcode 5.1. Enjoy!

About the Author Jon Reid

Jon is a consultant on Clean Code for iOS, focusing on Test Driven Development, unit testing, refactoring, and design. He's been practicing TDD since 2001. You can learn more about his background, or see what services he can bring to your organization.

follow me on:
Disclosure: The book links below are affiliate links. If you buy anything, I earn a commission, at no extra cost to you.
  • Jason J. W. Williams says:

    After updating according to the instructions above, my Test target still runs coverage on every invocation, I don’t get a prompt for coverage or not.

    • Derek Knight says:

      It could be due to the space in your folder name (which you mention in another response). I had the same problem until I removed the space in my folder name. FWIW I tried to rework the scripts so they’ve be more space tolerant, but just dug myself into more of a hole (bash is not my strong suit)

    • Jon Reid says:

      Any luck on this? BTW there have been a few patches to XcodeCoverage recently.

  • Roman says:

    Hey!
    Your tool is awesome, thank you a lot.

    Do you know if coverage files generated by Xcode can support branch coverage? Tried adding key
    –rc lcov_branch_coverage=1
    to lcov calls and didn’t get anything, assuming that coverage files don’t provide that info.

  • Sana says:

    Hi,

    I have followed the steps you have mentioned above for Xcode 5.1 but am getting the following:

    /x86_64/iOS_MobileSDK.gcda!
    Finished .info-file creation
    Reading tracefile Coverage.info
    lcov: ERROR: no valid records found in tracefile Coverage.info
    Reading tracefile Coverage.info
    lcov: ERROR: no valid records found in tracefile Coverage.info
    Reading data file Coverage.info
    genhtml: ERROR: no valid records found in tracefile Coverage.info

    thanks,

    Sana

  • Andy says:

    Love the tool, thanks a lot.

    I see gcda and gcno files in two different directories. One directory is named appName.build/ and the other is appNameTests.build/ (basically each target has its own directory). There are gcda/gcno files for every file in the project, duplicated in each directory. The only thing is that the coverage metrics captured in those files are vastly different. So my question is, which metrics are to be believed? What is the difference in those files?

    Just trying to raise some visibility to this question. I also posted it as an issue on XcodeCoverage in github, and here’s a reference to the same question on stackoverflow

    http://stackoverflow.com/questions/22188093/app-builds-gcda-gcno-files-vs-apptests-builds-gcda-gcno-files

    Thanks, and keep testing

  • Andy says:

    Another question – I often follow the pattern where I use OCMock’s partial mock objects, to partially mock the class under test (the first section on this blog shows a write up of the pattern https://engineering.aweber.com/improving-ios-unit-tests-with-ocmock/).

    The problem though, is that using XcodeCoverage, it shows 0 lines covered on any non-mocked method invoked on a partial mock. Obviously I know that not to be the case. Any ideas what’s up? CoverStory shows the same issue.

    I’m loading OCMock via a cocoapod if that makes a difference.

  • Anonymous says:

    I can generate code coverage but only partially. I always get segmentation fault. :(
    XcodeCoverage/llvm-cov-wrapper.sh: line 13: 15665 Segmentation fault: 11 /usr/bin/gcov $*

    Does anyone have the same issue?

  • Thanks for keeping things up to date – I love this particular tool and use it regularly.

    One nit – it may be true that the _gov_flush() workaround isn’t required if your project is based on Apple’s newer XCTest framework, but if your test code is still based on the older OCUnit framework, it does still appear to be necessary.

  • Jason J. W. Williams says:

    Also, it still doesn’t handle paths with spaces properly.

  • Jason J. W. Williams says:

    Does anyone else see a ton of these statements in your build output when generating coverage data with your tests:

    profiling:invalid arc tag (0x726f7461)
    profiling:invalid arc tag (0x00af57e9)
    profiling:invalid arc tag (0x00000012)
    profiling:invalid number of counters (10)

    • Derek Knight says:

      Looks like we are running across the same problems. If you get any results, please add them as comments here.

    • Derek Knight says:

      I was getting this too, then I did a clean and rebuild. The message has gone away. I think it could have been related to calls to __gcov_flush, which I’ve now removed. Apparently they are not needed if you’re using XCTest.

  • Fernando Canon says:

    Since upgrade to xcode 5.1.1 Im getting a crash on OCMockito, when calling methods from a mock object. This is the console message:


    [NSInvocation mkt_arrayArguments]: unrecognized selector sent to instance 0x2864db0
    :0: error: -[ExampleTests testUsingNumbers] : -[NSInvocation mkt_arrayArguments]: unrecognized selector sent to instance 0x2864db0
    (
    0 CoreFoundation 0x00b1d5e4 __exceptionPreprocess + 180
    1 libobjc.A.dylib 0x007958b6 objc_exception_throw + 44
    2 CoreFoundation 0x00bba903 -[NSObject(NSObject) doesNotRecognizeSelector:] + 275
    3 CoreFoundation 0x00b0d90b ___forwarding___ + 1019
    4 CoreFoundation 0x00b0d4ee _CF_forwarding_prep_0 + 14
    5 ExampleTests 0x0279107a -[MKTInvocationMatcher setExpectedInvocation:] + 303
    6 ExampleTests 0x027901f9 -[MKTInvocationContainer setInvocationForPotentialStubbing:] + 150
    7 ExampleTests 0x0279251d -[MKTBaseMockObject prepareInvocationForStubbing:] + 52
    8 ExampleTests 0x027921b9 -[MKTBaseMockObject forwardInvocation:] + 79
    9 CoreFoundation 0x00b0d6da ___forwarding___ + 458
    10 CoreFoundation 0x00b0d4ee _CF_forwarding_prep_0 + 14
    11 ExampleTests 0x0278c7c3 -[ExampleTests testUsingNumbers] + 147
    12 CoreFoundation 0x00b11d1d __invoking___ + 29
    13 CoreFoundation 0x00b11c2a -[NSInvocation invoke] + 362
    14 XCTest 0x201032bf -[XCTestCase invokeTest] + 212
    15 XCTest 0x2010338d -[XCTestCase performTest:] + 111
    16 XCTest 0x2010417c -[XCTest run] + 82
    17 XCTest 0x20102a44 -[XCTestSuite performTest:] + 139
    18 XCTest 0x2010417c -[XCTest run] + 82
    19 XCTest 0x20102a44 -[XCTestSuite performTest:] + 139
    20 XCTest 0x2010417c -[XCTest run] + 82
    21 XCTest 0x20102a44 -[XCTestSuite performTest:] + 139
    22 XCTest 0x2010417c -[XCTest run] + 82
    23 XCTest 0x20105aa1 +[XCTestProbe runTests:] + 183
    24 libobjc.A.dylib 0x007a7737 +[NSObject performSelector:withObject:] + 70
    25 xctest 0x0000233e xctest + 4926
    26 xctest 0x00002590 xctest + 5520
    27 xctest 0x00002671 xctest + 5745
    28 xctest 0x00002007 xctest + 4103
    29 libdyld.dylib 0x01655701 start + 1
    )

    I´ve been struggling with it for a couple of days .. change back xcode version, install the framework from cocoa pods, install previos versions, and so on. But nothing
    Any Ideas??

  • Craig Fisher says:

    Am I supposed to implement these changes ONTOP of the changes that you outline here: http://qualitycoding.org/xcode51-code-coverage/#more-1538?

    The problem that I’m running into is that nothing seems to happen after I click the okay button to generate the code report.

    Not getting any linker errors or anything like. that. Kinda confused at what I missed. Everything is installed into the right directory, i’ve installed all the command line tools, etc.

    Thanks in advance to any insight you might have!

    -Craig

  • ted says:

    since upgrade to xcode 5.1, when I analyze the coverage data, I got an error like this:
    gcov(25639,0x7fff72849310) malloc: *** error for object 0x101002d40: pointer being realloc’d was not allocated
    *** set a breakpoint in malloc_error_break to debug
    Stack dump:
    0. Program arguments: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/gcov /Users/ted/Desktop/iOSCodeDebug/Coverage/Coverage_armv7s/CoverageTest_armv7s/20140527155010/SongDBAdapter.gcda -o /Users/ted/Desktop/iOSCodeDebug/Coverage/Coverage_armv7s/CoverageTest_armv7s/20140527155010
    /Users/ted/Desktop/iOSCodeDebug/lcov/XcodeCoverage/llvm-cov-wrapper.sh: line 13: 25639 Abort trap: 6 /usr/bin/gcov $*

    I don’t know why this error occured,do you have any ideas?

  • Ryan says:

    First, thanks for providing this, looks quite useful as I’ve seen some reports from it before. Am now trying to get code coverage working for a library target following the steps that were outlined. In Xcode 5.1 it says:

    . . ./Build/Products/Debug-iphonesimulator ~/workspace/sdk/ios/x/XcodeCoverage ~/workspace/sdk/ios/x/XcodeCoverage
    Capturing coverage data from . . ./Build/Intermediates/x.build/Debug-iphonesimulator/x.build/Objects-normal/x86_64
    Found gcov version: 4.2.1
    Scanning . . ./Build/Intermediates/x.build/Debug-iphonesimulator/x.build/Objects-normal/x86_64 for .gcda files …
    geninfo: ERROR: no .gcda files found in . . ./Build/Intermediates/x.build/Debug-iphonesimulator/x.build/Objects-normal/x86_64!
    Reading tracefile Coverage.info
    lcov: ERROR: no valid records found in tracefile Coverage.info
    Reading tracefile Coverage.info
    lcov: ERROR: no valid records found in tracefile Coverage.info
    Reading data file Coverage.info
    genhtml: ERROR: no valid records found in tracefile Coverage.info

    The .gcno files are there but the .gcda files are not. For the i386 build, there are both .gcno and .gcda files.

    Steps:
    Instrument Program Flow Yes for debug
    Generate Test Coverage Files Yes for debug
    Added XcodeCoverage/exportenv.sh to Run Script of library target
    Run unit tests (kiwi) with Cmd+U. Unit tests are a separate target but rely on the library target I’m trying to get coverage on.

    I tried adding:
    -lgcov to Other Linker Flags
    Set “Application does not run in background” to Yes in the Unit tests target

    Not sure what else to try. Anyone have any advice?

  • Konrad says:

    I wonder if code coverage can (and how?) include Development Pods folder. For now when I’m using standard way (described here) I get the report which doesn’t include *.m files from Pod (just *.h file).
    The second problem is to find a way to have code coverage only for methods called by written Unit Tests, and not all methods called by system during testing.

    • Jon Reid says:

      Konrad,

      Yes, you can add coverage for pods and other nested projects.

      First, you have to track down the object directory directory for the project you want to add.

      Then edit envcov.sh, adding a definition for the new directory. For example:


      # Added shared lib
      LIB_OBJ=${OBJROOT}/ANJShared.build/Debug-iphonesimulator/ANJShared.build/Objects-normal/${NATIVE_ARCH}

      Finally, edit getcov and add some lines to gather_coverage(). What you want to do is capture the data to a different tracefile, then add all the tracefiles together. For example:


      gather_coverage()
      {
      LCOV --capture --derive-func-data -b "${SRCROOT}" -d "${OBJ_DIR}" -o ${LCOV_INFO}

      # Added shared lib
      LCOV --capture --derive-func-data -b "${SRCROOT}" -d "${LIB_OBJ}" -o Shared.info
      LCOV -a ${LCOV_INFO} -a Shared.info -o ${LCOV_INFO}
      }

  • yueer says:

    Seems that the “env.sh” file was missing?

  • Adam Hall says:

    Apple has their own ‘how-to’ they released in March 2014 (according to Google’s search results) that explains how to hook GCOV.

    https://developer.apple.com/library/ios/qa/qa1514/_index.html

    • Jon Reid says:

      Thanks, Adam! Some of that information is now dated, while other parts are new to me.

      • maggie says:

        Jon, thanks for sharing this. Our team has been working on Xcode 6.1 with IOS 8 sdk, supporting both 32-bit and 64-bit. According to this article on apple’s website(https://developer.apple.com/library/ios/qa/qa1514/_index.html), I just set the UIApplicationExitsOnSuspend key to YES to force coverage data to be written out to .gcda files on mobile. But we have more than 2600 .gcno files in our application, so there are always some .gcda file cannot be written out in the short time. Another issue is that the output .ipa with coverage information cannot be installed on some 32-bit iphone. Any idea how to solve this? thank you very much!

        • Jon Reid says:

          Maggie, that document is dated; there’s no longer any need to force the data to be flushed.

          Are you trying to get coverage information on real devices instead of the simulator? If so, what is actually driving the app that you want to measure?

  • Vincent Le Bourlot says:

    Hi! thanks for that tool, it looks very promising for my work. I’ve been using gcov following apple’s method linked by Adam, and visualizing the reports with CoverStory. But I would like to be able to generate html reports for portability reasons. Unfortunately, I can’t use lcov. I get the following error every time I try to use geninfo:

    geninfo: ERROR: myfile.gcno: found unrecognized record format

    I’m running OS X 10.9.5 with all the last updates including Xcode 6 and the last version of the Command Line Tools. Any idea how to solve that?
    Thanks a lot!

    • maggie says:

      I’ve been encountered the same issue with you. There are more than 2600 .gcno files in our application, and it generates the error (geninfo: ERROR: myfile.gcno: found unrecognized record format) for about 100 .gcno files. Did you ever resolve this?

  • Ben Smith says:

    Hi Jon,

    I’m in the process of separating out parts of quite a large app into separate libraries which will be loaded via CocoaPods. I’ve set up these libraries using the pod lib create command, which creates a library from a template, with an Example project set up all ready to use the library. Tests you write are actually written against the example project. With this setup I’m struggling to figure out how to set up XcodeCoverage. If I do it normally, I end up just showing the coverage of the files in the Example project, instead of the coverage of the actual library that I’m writing. Do you have any insight into how I could alter the exportenv.sh file to properly show coverage of my library instead of the Example project?

    Sincerely,

    Ben.

    • Vig says:

      I am looking for this as well. My library is on development pods and I was wondering how to get the test coverage for the library instead of the example project.

  • >