PIT line coverage excludes containerised integration tests

[astrod0m]

Characteristics

• Issue Type: [test report]
• Reproducibility: [always]
• Severity: [major]
• Tool/Service/Component: [pitmp-maven-plugin:1.3.7]
• Execution Environment: [n/a]
• Reporter: []

Description

Context: I've enabled PIT for a Java project with containerised integration tests. The test containers are instrumented to output the coverage from the integration tests at target/jacoco.exec before shutdown. The project already has Jacoco coverage enabled, which takes into consideration coverage from both unit and integration tests. PITMP has a simple integration with the jacoco-maven-plugin, with no custom configuration. The integration tests are picked up by PITMP (after setting parseSurefireConfig to false).

Problem: The line coverage calculated by PIT is missing some lines which are marked as covered in the existing Jacoco analysis, which leads me to suspect that PIT is unable to include the coverage from the containerised integration tests. This is supported by having some of those lines only covered in integration tests.

What I've tried: I looked into pointing the Jacoco plugin in PITMP to output the coverage to the same file as the containerised integration tests (via jacoco.destFile), and set it to append onto the existing file (via jacoco.append). Unfortunately there was no change in the line coverage reported by PITMP.

Additions to PITMP configuration: <jvmArgs> <jvmArg>-Djacoco.destFile=${project.build.directory}/jacoco.exec</jvmArg <jvmArg>-Djacoco.classDumpDir=${project.build.directory}</jvmArg> <jvmArg>-Djacoco.append=true</jvmArg> <jvmArg>-Djacoco.skip=true</jvmArg> <jvmArg>-Djacoco.dumpOnExit=true</jvmArg> <jvmArg>-Djacoco.output=file</jvmArg> </jvmArgs>

Questions:

1. Where does PITMP store the jacoco.exec file? I couldn't see any trace of it in the project after a successful PITMP run.
2. Are the Jacoco parameters passed as jvmArgs to PITMP successfully set on the Jacoco plugin in PITMP?
3. Is it possible for PIT to rely on a preexisting Jacoco coverage file, skipping its own line coverage?

Steps to reproduce

Other files and URLs

Relationships

Help on issue template

Preview to follow the link or open file .github/ISSUE_DOC.md