Generating LCOV Coverage with QtCreator

While QtCreator is an excellent IDE for Qt projects it has no built-in support for unit testing and test coverage. In that article I’ll show you how to unit test your Qt application, automatically generate LCOV html coverage report and view them in a browser with just one click.

Preparations

First of all I assume you have the gcov library as well as lcov and genhtml applications installed on your system. Next you need to tell QtCreator how to compile and link against that library. You do that by adding the following lines to the pro file of your unit test project:

The second step is to download my processCoverage.sh bash script from github. Essentially it analyzes recursively a path for coverage data provided by the gcov library. That data is created when programs linked against the gcov library are run. The script processes that coverage data using lcov and creates the html report using genhtml.

Putting The Pieces Together

The goal is to just click the QtCreator’s Run button to start the unit test, generate the coverage report and have it shown in a browser. To do that you need to add a custom run configuration to the unit test project. The following screenshot illustrates the steps:

QtCreator Run Settings

  1. Select the Project button on the left sidebar.
  2. Click to the Run tab to switch to the Run Settings.
  3. From the Add drop down menu select Custom Executable and make sure the executable and working directory are correct. Also you should check Run in terminal to see the output of failed tests and coverage processor.
  4. In the Arguments field you tell QTestLib (Qt 4.8 | Qt 5.0) to produce xml output which will give you in the terminal just the line of how many tests failed as an indicator for why coverage data might not be available or incomplete. Also the processCoverage.sh script is called with a file pattern and finally the index.html of the coverage report is sent to a browser which is Opera in that example.

    Here’s a template of the argument line. You need to insert the paths to the gnco files created by gcov, to the processCoverage.sh script, to the location where you want the lcov report generated to, your project name, and your browser:

    An example in full length:

With A Single Click You Get

Now, when you click the Run button in QtCreator a terminal appears printing in the second line the number of failed tests followed by lcov logs.

Terminal Output Of QtCreator Run With Coverage Generator

Once LCOV is finished the coverage report will be opened in your web browser.

Coverage Report Shown In Browser

Thanks for reading. Feedback is welcome. :-)

Edit:
ZIP of a simple Qt 4.8.4 example project: LCOVblog.tar.gz
Edit (2014-02-06):
Changed links of gists linking to the new improved argument line for complex project structures.

8 comments to Generating LCOV Coverage with QtCreator

  • Gabe

    Cool, does this work with all OS?

    • robert

      It should also work for Mac as the underlying system is unix like. You might need to change the hash bang in the processCoverage.sh script to point to the shell that’s installed on MacOS (if it’s not bash).

      For Windows it should work, too. You’d need the genhtml, lcov tools as well as a gcov.dll (or something similar). Besides that, you’ll need to translate the processCoverage.sh script to a bat file or install bash in windows. And you need to call that script in the arguments field of the custom run configuration, of course.

      In case you translate the script to Windows or Mac you’re welcome to contribute it to my processCoverage github repository.

  • Hendrik

    Hi Robert,

    thank you for this great post. I managed to get it running on my PC but unfortunately I have some issues with it.

    First:
    When I run it, my first output line on the console is “Capturing coverage data from .” I don’t have any output from my original program. In your screenshot I can see that your first line is “Failed tests: 0″. Any ideas what options I missed to have the output from the original program on the console as well?

    Second:
    Everything runs great when I have all source files in the same folder. I can run it repeatedly without problems. As soon as I have subfolders that contains the source files (for example /src/folderA, /src/folderB, etc), I can run it without problems once. If I run it a second time it fails to create the new code coverage information. The difference on the console output begins after the line “Reading tracefile coverage.info”. Normally the next lines are “Extracting /…/a.cpp” etc. My output is: “Extracted 0 files”
    “Writing data to coverage-stripped.info”
    “Overall coverage rate”
    “lines……: no data found”

    If I delete the “html-coverage” folder. It runs normal again once and the same problem occurs again when I try a second run.
    Any ideas what the problem might be?

    I am using Qt 4.8.1 on an Ubuntu 12.04 System.

  • robert

    Hi Hendrik,

    Originally I wrote the post using Qt 5.0.0 on Kubuntu 13.04. But I just got it to work as expected with Qt 4.8.4.

    To first:
    That line is generated in my testMain.cpp via qDebug() < < "Failed tests: " << result;

    To second:
    Strange. I just checked with installed Qt 4.8.4 on Kubuntu 13.04. A simple sample project with a test library where the class to be tested is in a subfolder. It works well on my machine. I zipped the project folders together, so you can check pro files against yours. The ZIP will be attached to the blog post above at the bottom.

    The QtCreator run configuration belonging to the ZIP for my system is:
    Command:
    /home/robert/devel/qt4.8/LCOVblog/libBlogLcov48Test-build-Desktop_Qt_4_8_4_GCC_64bit_SDK-Debug/LibBlogLcov48Test
    Arguments:
    -xml > testresults.out && /home/robert/bin/processCoverage.sh “*/libBlogLcov48/src/*” && /usr/bin/opera html-coverage/index.html &
    Working Directory:
    /home/robert/devel/qt4.8/LCOVblog/libBlogLcov48Test-build-Desktop_Qt_4_8_4_GCC_64bit_SDK-Debug

    Could you please check if the ZIPed project works on your system as described in my blog post?

    Rob

    • Hendrik

      To first:
      With qDebug, it works for me as well. But why is it working with qDebug but not with std::cout?

      To second:
      Now its getting really strange. With your project, I have the same problem as above immediately, not just after the first run.
      I also updated to Qt 5 and updated to the newest qt creator version. Doesn’t make any difference.

      For my project I created a work around by adding “rm -rf html-coverage” at the beginning of the script. But its still strange why it works perfect for some projects and not at all for the other.

      • Hendrik

        alright, I found the solution. First of all, your project runs without problems. I just copied the line from above and changed the paths. Unfortunately the ” was different from the one you enter in the argument field in qtCreator when you don’t copy it. So just a wrong character, my bad.

        Regarding my original problem: It seems my project structure caused the trouble. My structure looked like this:
        \gcovTest
        …\build_Debug
        …\gcovTest (containing the main.cpp and gcovTest.pro)
        …\src
        ……\folderA (containing a.h and a.cpp)
        ……\folderB (containing b.h and b.cpp)

        I changed it so the src folder is inside the inner gcovTest folder:
        \gcovTest
        …\build_Debug
        …\gcovTest (containing the main.cpp and gcovTest.pro)
        ……\src
        ………\folderA (containing a.h and a.cpp)
        ………\folderB (containing b.h and b.cpp)

        And now it works without trouble.
        To have all std::cout from the original program on the console as well, I removed “> testresults.out” from the arguments:

        changed:
        -xml > testresults.out && /home/robert/bin/processCoverage.sh “*/libBlogLcov48/src/*” && /usr/bin/opera html-coverage/index.html &

        to:
        -xml && /home/robert/bin/processCoverage.sh “*/libBlogLcov48/src/*” && /usr/bin/opera html-coverage/index.html &

        Thx a lot for your help and again for this great post!

        Greets

        Hendrik

  • robert

    Hi Hendrik,

    great that you got it working! :-D

    Rob

  • [...] posted about “Generating LCOV Coverage with QtCreator” already. That is pretty nice as it opens the coverage report in a browser right after running the [...]

Leave a Reply

  

  

  

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>