[Question] can we use multiple v8-json files to merge coverage?

[stevez]

In order to merge multiple v8 coverage report, does monocart support merging json files instead of raw files like nyc for Istanbul?

[cenfun]

The v8-json you said is generated by NODE_V8_COVERAGE?
If so, we can use option dataDir (not inputDir)

{
  // ...
  dataDir: "./coverage/v8-coverage"
  // ...
}

see example: https://github.com/tstyche/tstyche/blob/main/scripts/report-coverage.js

Also there is API addFromDir

await mcr.addFromDir("./coverage/v8-coverage");
await mcr.generate();

The MCR will automatically merge all the added coverage data when executing generate()

await mcr.addFromDir("./coverage/v8-coverage1");
await mcr.addFromDir("./coverage/v8-coverage2");
await mcr.generate();

[stevez]

V8-json is one of the format your report provided. What I mean is I found using raw files to merge is a little bit complicated: it still need to scan the source folder, and the config for merging report needs to combine both playwright report config and jest report config. So I am thinking if jest and playwright both generate v8 json files, and both include all files information, it would be great if we can just merge 2 json files

[cenfun]

I see, you are using monocart-reporter + Playwright.
The monocart-reporter is a test report but integrates MCR as coverage report.
When merging the test reports, the coverage reports will be merged automatically if we specify the raw report in coverage options. (since version monocart-reporter v2.7.0)
I don't think there's a better way to do the merging currently.

However, perhaps we could try using independent MCR to collect and generate coverage reports directly (without monocart-reporter).
see example https://github.com/cenfun/playwright-coverage

[cenfun]

BTW, if you are using Github Actions or similar CI/CD workflow/piplines like Jenkins, there should be a way to upload and download the artifacts in a job, we can easily transfer and use these artifacts (zip all json files) to merge reports.

see GithubActions example: https://github.com/cenfun/merge-code-coverage/blob/main/.github/workflows/main.yml

[stevez]

Could you explain details how your example can help?

My use case is exactly same as this: https://github.com/cenfun/merge-code-coverage

I wonder if we can simply merge the generated coverage-report.json files

[cenfun]

No, We can not merge with the generated coverage-report.json files
The coverage-report.json is the data file generated for HTML report (V8 only), It does not include the original coverage range data which is used for the merging algorithm.
We can only merge with the raw data (raw report data or raw V8 coverage data), it includes all the coverage info we need.

[cenfun]

In fact, merging coverage data is very complex. Even for Istanbul nyc, there are still some issues with merging

• Merging coverage files doesn't produce accurate cumulative result istanbuljs/nyc#1302
• Merging with nyc storybook test coverage with other reports generated by coverageProvider 'istanbul' result in wrong coverage istanbuljs/nyc#1559
• Jest and Cypress's coverage merged results is wrong  istanbuljs/nyc#1533
• Branch coverage underreported when merging Jest shards istanbuljs/nyc#1581

The possible reason is that different versions of istanbul-lib-instrument or related compiler lead to variations in the coverage data. At this point, it should be an advantage for the V8 format.

[stevez]

Got it, thank you