Scoverage HTML report generation fails

[leon-becker-secforge]

Hi,
after upgrading to mill 0.12.0 the HTML report for coverage cannot be generated anymore and shows the following error:

Trace

[14/14] Demo.scoverage.htmlReport
[14] Processing coverage data for 1 data locations
[14] [info] Found 1 subproject scoverage data directories [/mnt/c/Users/LeonAlexanderBeckers/Desktop/repr/out/Demo/scoverage/data.dest]
[14] Exception while building coverage report. No source root found for '/mnt/c/Demo/src/Main.scala' (source roots: '/mnt/c/Users/LeonAlexanderBeckers/Desktop/repr/Demo/src/')
[14] java.lang.RuntimeException: No source root found for '/mnt/c/Demo/src/Main.scala' (source roots: '/mnt/c/Users/LeonAlexanderBeckers/Desktop/repr/Demo/src/')
[14]    at scoverage.reporter.BaseReportWriter.relativeSource(BaseReportWriter.scala:35)
[14]    at scoverage.reporter.BaseReportWriter.relativeSource(BaseReportWriter.scala:23)
[14]    at scoverage.reporter.ScoverageHtmlWriter.classRow(ScoverageHtmlWriter.scala:249)
[14]    at scoverage.reporter.ScoverageHtmlWriter.$anonfun$classesTable$2(ScoverageHtmlWriter.scala:239)
[14]    at scala.collection.immutable.List.map(List.scala:247)
[14]    at scala.collection.immutable.List.map(List.scala:79)
[14]    at scoverage.reporter.ScoverageHtmlWriter.classesTable(ScoverageHtmlWriter.scala:239)
[14]    at scoverage.reporter.ScoverageHtmlWriter.overview(ScoverageHtmlWriter.scala:460)
[14]    at scoverage.reporter.ScoverageHtmlWriter.write(ScoverageHtmlWriter.scala:65)
[14]    at mill.contrib.scoverage.worker.ScoverageReportWorkerImpl.report(ScoverageReportWorkerImpl.scala:38)
[14]    at mill.contrib.scoverage.ScoverageReportWorker$$anon$3.report(ScoverageReportWorker.scala:65)
[14]    at mill.contrib.scoverage.ScoverageModule$ScoverageData.$anonfun$doReport$2(ScoverageModule.scala:138)
[14]    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
[14]    at mill.api.Result$.create(Result.scala:25)
[14]    at mill.contrib.scoverage.ScoverageModule$ScoverageData.$anonfun$doReport$1(ScoverageModule.scala:142)
[14]    at mill.define.Task$TraverseCtx.evaluate(Task.scala:215)
[14]    at mill.eval.GroupEvaluator.$anonfun$evaluateGroup$10(GroupEvaluator.scala:308)
[14]    at mill.eval.GroupEvaluator.$anonfun$evaluateGroup$9(GroupEvaluator.scala:302)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at mill.api.SystemStreams$.$anonfun$withStreams$6(SystemStreams.scala:100)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at mill.api.SystemStreams$.$anonfun$withStreams$5(SystemStreams.scala:99)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at mill.api.SystemStreams$.$anonfun$withStreams$4(SystemStreams.scala:98)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at scala.Console$.withErr(Console.scala:193)
[14]    at mill.api.SystemStreams$.$anonfun$withStreams$3(SystemStreams.scala:97)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at scala.Console$.withOut(Console.scala:164)
[14]    at mill.api.SystemStreams$.$anonfun$withStreams$2(SystemStreams.scala:96)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at scala.Console$.withIn(Console.scala:227)
[14]    at mill.api.SystemStreams$.$anonfun$withStreams$1(SystemStreams.scala:95)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at mill.api.SystemStreams$.withStreams(SystemStreams.scala:94)
[14]    at mill.eval.GroupEvaluator.$anonfun$evaluateGroup$8(GroupEvaluator.scala:299)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at mill.eval.GroupEvaluator.wrap$1(GroupEvaluator.scala:298)
[14]    at mill.eval.GroupEvaluator.$anonfun$evaluateGroup$3(GroupEvaluator.scala:307)
[14]    at mill.eval.GroupEvaluator.$anonfun$evaluateGroup$3$adapted(GroupEvaluator.scala:256)
[14]    at scala.collection.immutable.Vector.foreach(Vector.scala:2125)
[14]    at mill.eval.GroupEvaluator.computeAll$1(GroupEvaluator.scala:256)
[14]    at mill.eval.GroupEvaluator.evaluateGroup(GroupEvaluator.scala:334)
[14]    at mill.eval.GroupEvaluator.$anonfun$evaluateGroupCached$1(GroupEvaluator.scala:202)
[14]    at mill.api.Logger.withPrompt(Logger.scala:83)
[14]    at mill.api.Logger.withPrompt$(Logger.scala:81)
[14]    at mill.util.PrefixLogger.withPrompt(PrefixLogger.scala:7)
[14]    at mill.eval.GroupEvaluator.evaluateGroupCached(GroupEvaluator.scala:67)
[14]    at mill.eval.GroupEvaluator.evaluateGroupCached$(GroupEvaluator.scala:47)
[14]    at mill.eval.EvaluatorImpl.evaluateGroupCached(EvaluatorImpl.scala:16)
[14]    at mill.eval.EvaluatorCore.$anonfun$evaluate0$2(EvaluatorCore.scala:163)
[14]    at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
[14]    at mill.eval.ExecutionContexts$ThreadPool$$anon$1.$anonfun$run$3(ExecutionContexts.scala:69)
[14]    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at mill.api.SystemStreams$.$anonfun$withStreams$6(SystemStreams.scala:100)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at mill.api.SystemStreams$.$anonfun$withStreams$5(SystemStreams.scala:99)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at mill.api.SystemStreams$.$anonfun$withStreams$4(SystemStreams.scala:98)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at scala.Console$.withErr(Console.scala:193)
[14]    at mill.api.SystemStreams$.$anonfun$withStreams$3(SystemStreams.scala:97)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at scala.Console$.withOut(Console.scala:164)
[14]    at mill.api.SystemStreams$.$anonfun$withStreams$2(SystemStreams.scala:96)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at scala.Console$.withIn(Console.scala:227)
[14]    at mill.api.SystemStreams$.$anonfun$withStreams$1(SystemStreams.scala:95)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at mill.api.SystemStreams$.withStreams(SystemStreams.scala:94)
[14]    at mill.eval.ExecutionContexts$ThreadPool$$anon$1.$anonfun$run$2(ExecutionContexts.scala:69)
[14]    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
[14]    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[14]    at mill.eval.ExecutionContexts$ThreadPool$$anon$1.run(ExecutionContexts.scala:68)
[14]    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
[14]    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[14]    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[14]    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[14]    at java.base/java.lang.Thread.run(Thread.java:840)
[14/14] ======================================================================================================================== Demo.scoverage.htmlReport ========================================================================================================================= 1s
1 tasks failed
Demo.scoverage.htmlReport java.lang.RuntimeException: No source root found for '/mnt/c/Demo/src/Main.scala' (source roots: '/mnt/c/Users/LeonAlexanderBeckers/Desktop/repr/Demo/src/')
    scoverage.reporter.BaseReportWriter.relativeSource(BaseReportWriter.scala:35)
    scoverage.reporter.BaseReportWriter.relativeSource(BaseReportWriter.scala:23)
    scoverage.reporter.ScoverageHtmlWriter.classRow(ScoverageHtmlWriter.scala:249)
    scoverage.reporter.ScoverageHtmlWriter.$anonfun$classesTable$2(ScoverageHtmlWriter.scala:239)
    scala.collection.immutable.List.map(List.scala:247)
    scala.collection.immutable.List.map(List.scala:79)
    scoverage.reporter.ScoverageHtmlWriter.classesTable(ScoverageHtmlWriter.scala:239)
    scoverage.reporter.ScoverageHtmlWriter.overview(ScoverageHtmlWriter.scala:460)
    scoverage.reporter.ScoverageHtmlWriter.write(ScoverageHtmlWriter.scala:65)
    mill.contrib.scoverage.worker.ScoverageReportWorkerImpl.report(ScoverageReportWorkerImpl.scala:38)
    mill.contrib.scoverage.ScoverageReportWorker$$anon$3.report(ScoverageReportWorker.scala:65)
    mill.contrib.scoverage.ScoverageModule$ScoverageData.$anonfun$doReport$2(ScoverageModule.scala:138)
    scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)

I am using WSL2 under Windows 11, but the exception also happens in the default Linux runner on GitHub.

When looking into the generated coverage files, I suspect this to be caused by the newly introduced sandboxing and changing the working directory of one of the used processes.

coverage files

Note: build.sc was located at /mnt/c/Users/LeonAlexanderBeckers/Desktop/repr and is using the files given in the reproduction section.

data.dest/scoverage.coverage under 0.12.0

../../../../Demo/src/Main.scala
<empty>
Demo
Object
<empty>.Demo
demo
17
25
2
demo
DefDef
false
0
false
def demo


data.dest/scoverage.coverage under 0.11.8

Demo/src/Main.scala
<empty>
Demo
Object
<empty>.Demo
demo
17
25
2
demo
DefDef
false
0
false
def demo


Steps reproduce

1. Create the files described below.
2. Run mill Demo.test
3. Run mill Demo.scoverage.htmlReport => we see the error
4. Change .mill-version to 0.11.8
5. Run mill Demo.test
6. Run mill Demo.scoverage.htmlReport => we see no error

Code files

.mill-version

0.12.0

build.sc

import mill._, scalalib._

import $ivy.`com.lihaoyi::mill-contrib-scoverage:`

import mill.contrib.scoverage.ScoverageModule

object Demo extends ScalaModule with ScoverageModule {
  def scalaVersion   = "3.5.2"
  
  def scoverageVersion = "2.2.1"

  def zioTestVersion = "2.1.11"
  
  object test extends ScoverageTests with ScalaTests {
    def ivyDeps = Agg(
      ivy"dev.zio::zio-test:$zioTestVersion",
      ivy"dev.zio::zio-test-sbt:$zioTestVersion"
    )
	
	def testFramework = "zio.test.sbt.ZTestFramework"
  }
}

Demo/src/Main.scala

object Demo {
  def demo = "demo"
}

Demo/test/src/Main.scala

package demo

import zio.test.*

object Test extends ZIOSpecDefault {
  def spec = test("demo")(
    assertCompletes
  )
}

[kmudrick]

Yeah, coverage is busted with scala 3. I ran into this last week (simply changing the scala version from 3.x to 2.13.x in the build makes it work)

In scala 3, coverage is actually computed/generated by scala itself, so the coverage files having the wrong path seems to point to maybe bad path related args being passed to the compiler? I have no idea.

[kmudrick]

Note: this affects the xml report and cobertura xml report as well.

[leon-becker-secforge]

The console report seems to be unaffected, but it only prints the percentages.

Scala 3 is not the problem here. The coverage report was generated just fine by Scala 3.3.3 and mill 0.11.8 before I did the update. Now with mill 0.12.0 the report fails no matter if I used Scala 3.3.3 or 3.5.2.

In 7da0265 the paths passed to the report generation were changed from os.Path to Java Path, this could be the cause of the problem.

[kmudrick]

@leon-becker-secforge Sorry - did not mean to imply that scala 3 itself is the problem, moreso that mill interfacing with scala3, etc is the problem. The same codebase and mill version will produce reports correctly with 2.x (specifically, it will produce the coverage file correctly, as it uses scoverage for that, whereas with scala3, mill only uses scoverage for the report generation portion).

[lihaoyi]

Yes it seems to be caused by the change in working directory from the workspace root to the .dest/sandbox/ folder. The code in question is https://github.com/com-lihaoyi/mill/tree/main/contrib/scoverage. Do you know who creates this scoverage.coverage file? Presumably there should be an override or something that we can use to pass in T.workspace

[kmudrick]

Hmm, accoding to the dotty docs on coverage:

You can also set -sourceroot:PATHS_ROOT to customize how the path of your source files are resolved. Note that -sourceroot also sets the root path of the SemanticDB files.

That doc also links to the original scala 3 PR that brought in instrumentation.