Skip to content
This repository has been archived by the owner on Dec 21, 2023. It is now read-only.

Measure coverage using codecov #77

Merged
merged 14 commits into from
Oct 2, 2018
Merged

Measure coverage using codecov #77

merged 14 commits into from
Oct 2, 2018

Conversation

bassosimone
Copy link
Member

@bassosimone bassosimone commented Oct 2, 2018

We were using coveralls. The best client for C++ seems to be cpp-coveralls, which:

  1. does not work with Python 3.3, which is what we have in Ubuntu 18.04

  2. does not work with Debian testing even though this issue has been fixed in master

  3. even when using the master branch in Debian testing, it shows other issues that I could not easily understand or fix

At the same time, using an older distribution image is problematic because the code in here is assuming that we use OpenSSL 1.1.0+.

I spent a fair amount of time trying to fix this issue in several ways, then I decided to give up and use a much better strategy:

  1. measure the coverage using lcov which is more likely to be up-to-date with gcc format than cpp-coveralls since it's developed by the GNU project

  2. use codecov that allows to submit a .lcov file

  3. also, use codecov because their client is written in bash, hence it does not suffer of all the issues that plague Python dependency management (long live, Go static binaries!).

As a note for the future self, it was also theoretically possible to avoid the lcov step, except that there is more than a single test binary, hence there are multiple coverage measurements. The default behaviour of codecov.bassh is such that it only takes one of these measurements, hence reporting a wrong coverage measurement (here's a place where cpp-coveralls is clearly doing better than codecov). This issue has been discussed, e.g., in JoakimSoderberg/coveralls-cmake#17. My solution to this problem is to let lcov (which should be in sync with gcc) do the dirty work of merging the various coverage reports and produce a consistent summary.

@bassosimone bassosimone added the bug Something isn't working label Oct 2, 2018
@codecov-io
Copy link

Codecov Report

❗ No coverage uploaded for pull request base (master@819b8e3). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##             master      #77   +/-   ##
=========================================
  Coverage          ?   87.96%           
=========================================
  Files             ?        5           
  Lines             ?     2725           
  Branches          ?        0           
=========================================
  Hits              ?     2397           
  Misses            ?      328           
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 819b8e3...b96594d. Read the comment docs.

@bassosimone bassosimone merged commit 07e4630 into master Oct 2, 2018
@bassosimone bassosimone deleted the x branch October 2, 2018 16:38
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants