Jarret.Byrne

Politics, Sociology, Firefighting, Technology, and Original Fiction

Gitlab CI PHPUnit Test Coverage Parsing

Test Coverage Parsing

If you use Gitlab CI (and I suggest you do if you are using Gitlab already), you may have noticed in the settings that there is an item titled “Test coverage parsing”.

It looks a little something like this:Screen Shot 2015-04-12 at 10.09.51 PM

This feature allows you to give Gitlab CI a regex that will parse the output of your test runner and grab what percentage of your code is covered by your unit tests. The resulting percentage will then display for each individual build in your list of builds.




I know for me, having this displayed in my face helps add that extra bit of incentive to write some more tests.

Parsing PHPUnit Output

If you want to do this for PHPUnit, first make sure that your runner script will output the code coverage information by running PHPUnit with the coverage-text flag:

Now that your runner is outputting the coverage percentage, you can put a regex in the aforementioned “Test coverage parsing” setting to grab the value. This regex is working for me so far:

If all goes well, you should start to see your code coverage percentage display for each successful build. Here is an example from my latest project:Screen Shot 2015-04-12 at 10.38.25 PM

Some Caveats

Two quick considerations:

  • All the machines that are running PHPUnit will need to also have xdebug:

  • If your tests fail, I do not think that the code coverage percentage will display for that build. If that is the case and you rerun the tests after fixing the problem, the coverage value will still not display for that particular build. Mildly aggravating, sure, but all the more reason to test locally before committing and pushing.

Next

What I Tell Them

8 Comments

  1. Marcos SF Filho

    Shouldn’t the regex have braces, like this:

    Lines:\s*(\d+.\d+\%)

    ?

    • jbyrne

      Hey Marcos,

      That definitely stands to reason – using a capturing group to define what you wish to return as your coverage value certainly makes sense, and I don’t think that it is a bad idea to do so for at least safety’s sake.

      That being said, it would appear that the way Gitlab CI parses out the coverage value is not through a capturing group. I have not played much with Ruby, but I believe that their code grabs from your output whatever matches your regex – the match, not the capture – and then from that string pulls out any value that then matches /\d+(\.\d+)?/. So the first number in your matched string will be what is returned, disregarding any captures.

      For reference, I found this in their code on 6/2/15, available here: https://gitlab.com/gitlab-org/gitlab-ci/blob/master/app/models/build.rb#L196

      If all that seems confusing, then, well, I agree. At least one other person does as well, as there is an open issue to make parsing coverage values easier: https://gitlab.com/gitlab-org/gitlab-ci/issues/124

      Hope this helps. Thanks for the comment!
      – JB

  2. Hi Jarret,

    Thanks for the blog post.

    Maybe it’s important to note that code coverage percentage parsing will not work if phpunit colors are enabled.

    Best regards,
    Davor

  3. Robert

    I’m using “\ \ Lines:\s*(\d+.\d+\%)” as the regular expression and it works quite well, even with colors enabled.

Leave a Reply

Powered by WordPress & Theme by Anders Norén