Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generated Clover XML logfiles do not validate against clover.xsd #578

Open
sebastianbergmann opened this issue Jan 16, 2018 · 8 comments
Open

Comments

@sebastianbergmann
Copy link
Owner

sebastianbergmann commented Jan 16, 2018

$ git clone [email protected]:sebastianbergmann/money.git

$ cd money

$ phpunit -c build

$ wget https://bitbucket.org/atlassian/clover/raw/bd0ebb6cc8cc3e3fdb878b2640078461b8902a32/etc/schema/clover.xsd

$ xmllint --noout --schema /tmp/clover.xsd build/logs/clover.xml 
build/logs/clover.xml:2: element coverage: Schemas validity error : Element 'coverage': The attribute 'clover' is required but missing.
build/logs/clover.xml:4: element package: Schemas validity error : Element 'package': This element is not expected. Expected is ( metrics ).
build/logs/clover.xml:2: element coverage: Schemas validity error : Element 'coverage': Missing child element(s). Expected is ( testproject ).
build/logs/clover.xml fails to validate
@sebastianbergmann
Copy link
Owner Author

This could be a breaking change.

@sebastianbergmann
Copy link
Owner Author

@sebastianbergmann
Copy link
Owner Author

Thanks to @jenschude for https://gist.github.com/jenschude/9619ba13a9e7743368ed07b21dad6179#gistcomment-2328139:

public class Foo
{
    public boolean bar(int a, int b)
    {
        if (a == b) {
            return false;
        }

        return true;
    }
}
import org.junit.Test;

import static org.assertj.core.api.Assertions.assertThat;

public class FooTest
{
    @Test
    public void one() {
        Foo foo = new Foo();

        assertThat(foo.bar(1, 2));
        assertThat(!foo.bar(1, 1));
    }
}
<?xml version="1.0" encoding="UTF-8"?>
<coverage generated="1516617047954" clover="4.2.0">
   <project name="foo" timestamp="1516617047145">
      <metrics coveredelements="6" complexity="2" loc="11" methods="1" classes="1" statements="3" packages="1" coveredconditionals="2" coveredmethods="1" elements="6" ncloc="10" files="1" conditionals="2" coveredstatements="3"/>
      <package name="default-pkg">
         <metrics coveredelements="6" complexity="2" loc="11" methods="1" classes="1" statements="3" coveredconditionals="2" coveredmethods="1" elements="6" ncloc="10" files="1" conditionals="2" coveredstatements="3"/>
         <file path="/Users/jensschulze/workspace/foo/src/main/java/Foo.java" name="Foo.java">
            <metrics coveredelements="6" coveredconditionals="2" complexity="2" loc="11" coveredmethods="1" methods="1" elements="6" classes="1" ncloc="10" statements="3" conditionals="2" coveredstatements="3"/>
            <class name="Foo">
               <metrics coveredelements="6" coveredconditionals="2" complexity="2" coveredmethods="1" methods="1" elements="6" statements="3" conditionals="2" coveredstatements="3"/>
            </class>
            <line complexity="2" visibility="public" signature="bar(int,int) : boolean" num="3" count="2" type="method"/>
            <line num="5" count="2" type="stmt"/>
            <line falsecount="1" truecount="1" num="5" type="cond"/>
            <line num="6" count="1" type="stmt"/>
            <line num="9" count="1" type="stmt"/>
         </file>
      </package>
   </project>
   <testproject name="foo" timestamp="1516617047145">
      <metrics coveredelements="4" complexity="1" loc="15" methods="1" classes="1" statements="3" packages="1" coveredconditionals="0" coveredmethods="1" elements="4" ncloc="11" files="1" conditionals="0" coveredstatements="3"/>
      <package name="default-pkg">
         <metrics coveredelements="4" complexity="1" loc="15" methods="1" classes="1" statements="3" coveredconditionals="0" coveredmethods="1" elements="4" ncloc="11" files="1" conditionals="0" coveredstatements="3"/>
         <file path="/Users/jensschulze/workspace/foo/src/test/java/FooTest.java" name="FooTest.java">
            <metrics coveredelements="4" coveredconditionals="0" complexity="1" loc="15" coveredmethods="1" methods="1" elements="4" classes="1" ncloc="11" statements="3" conditionals="0" coveredstatements="3"/>
            <class name="FooTest">
               <metrics coveredelements="4" complexity="1" methods="1" testruns="1" statements="3" coveredconditionals="0" coveredmethods="1" elements="4" testfailures="0" testduration="0.057999998331069946" conditionals="0" coveredstatements="3" testpasses="1"/>
            </class>
            <line complexity="1" visibility="public" signature="one() : void" num="8" testsuccess="true" count="1" testduration="0.057999998331069946" type="method"/>
            <line num="10" count="1" type="stmt"/>
            <line num="12" count="1" type="stmt"/>
            <line num="13" count="1" type="stmt"/>
         </file>
      </package>
   </testproject>
</coverage>

@RykHawthorn
Copy link

Hi, are you still working on it?

@jimdelois
Copy link

Is there an estimated timeline for this? Are there any works-in-progress that could use assistance to get this across the finish line, or can/should it be taken up anew?

There is a great deal of tooling that relies on correct Clover formatting - it is unfortunate that when using PHP with such tooling, the incompatibility with the Clover spec means that coverage features cannot be used (for example, AWS CodeBuild).

@sebastianbergmann
Copy link
Owner Author

I am not working on this and there is no timeline.

@vnrmc
Copy link

vnrmc commented Mar 25, 2021

@jimdelois use cobertura as small workaround (--coverage-cobertura, available since phpunit 9.4) and modify your buildpec:
... reports: codecoverage: ... file-format: 'COBERTURAXML'

@amottier
Copy link

amottier commented Aug 9, 2023

One issue that I have with current Clover report format is that the name attribute of file tag is actually used to store the full file path (If I understand correctly the code) whereas, according to XSD, it should only store the file name and the full file path should be stored in a path attribute (path attribute is not set in current report implementation).
Such behavior is confirmed by Clover Java implementation that set attributes values using only file name for the name attribute and absolute path for the path attribute.

In SonarQube, parsing of the report by Sonar PHP plugin has be done to handle correctly the current implementation that only include name attribute.
Also in SonarQube, the Clover plugin is using only the path attribute those making it incompatible with current implementation but supporting the XSD specification. (Edit: I realized that SonarQube Clover plugin is only targeting Java and Groovy).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants