Pitest should add all dependencies to forked classpath

[A248]

Using JUnit 5 and Maven surefire, it is common to declare the dependency on JUnit-Jupiter-Engine separately, so that test classes can't depend on Jupiter-Engine classes.

This is explained at https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit-platform.html . The example described there is:

<dependencies>
    [...]
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-api</artifactId>
        <version>5.3.0</version>
        <scope>test</scope>
    </dependency>
    [...]
</dependencies>
...
<build>
    <plugins>
        [...]
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <dependencies>
                <dependency>
                    <groupId>org.junit.jupiter</groupId>
                    <artifactId>junit-jupiter-engine</artifactId>
                    <version>5.3.2</version>
                </dependency>
            </dependencies>
        </plugin>
        [...]
    </plugins>
</build>

Relation to Pitest

This doesn't work with Pitest-maven. Pitest-maven collects the dependencies for itself and for the test-classpath.

However, Pitest-maven filters the forked classpath based on which dependencies are pitest plugins. This removes declarations of JUnit-Jupiter-Engine.

For example, use the following configuration:

			<plugin>
				<groupId>org.pitest</groupId>
				<artifactId>pitest-maven</artifactId>
				<version>1.6.7</version>
				<dependencies>
					<dependency>
						<groupId>org.pitest</groupId>
						<artifactId>pitest-junit5-plugin</artifactId>
						<version>0.14</version>
					</dependency>
					<dependency>
						<groupId>org.junit.jupiter</groupId>
						<artifactId>junit-jupiter-engine</artifactId>
						<version>${junit.version}</version>
					</dependency>
				</dependencies>
			</plugin>

which results in:

[INFO] Found shared classpath plugin : Default mutation engine
[INFO] Found shared classpath plugin : JUnit 5 test framework support
[INFO] Found shared classpath plugin : JUnit plugin
[INFO] Found shared classpath plugin : TestNG plugin
[INFO] Adding org.pitest:pitest-junit5-plugin to SUT classpath
[INFO] Adding org.pitest:pitest to SUT classpath
[INFO] Mutating from ${project.basedir}/target/classes

As shown, pitest and pitest-junit5-plugin are added to the SUT classpath, but junit-jupiter-engine is not added to the SUT classpath.

Symptoms

The classPathElements of ReportOptions includes the project, the test-scoped dependencies, and dependencies of pitest-maven which are pitest plugins. It should include all dependencies of pitest-maven, not just those which are pitest plugins.

Running report with ReportOptions [..., classPathElements=[project/target/test-classes, project/target/classes, ~/.m2/repository/org/junit/jupiter/junit-jupiter-params/5.7.1/junit-jupiter-params-5.7.1.jar, ~/.m2/repository/org/apiguardian/apiguardian-api/1.1.0/apiguardian-api-1.1.0.jar, ~/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.7.1/junit-jupiter-api-5.7.1.jar, ~/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar, ~/.m2/repository/org/junit/platform/junit-platform-commons/1.7.1/junit-platform-commons-1.7.1.jar, ~/.m2/repository/org/pitest/pitest-junit5-plugin/0.14/pitest-junit5-plugin-0.14.jar, ~/.m2/repository/org/pitest/pitest/1.6.7/pitest-1.6.7.jar], ...]

As a result, JUnit-Platform cannot find JUnit-Jupiter-Engine in the forked classpath. This causes Coverage generation minion exited abnormally! Here's the minion output running with verbose mode enabled: https://pastebin.com/Z2jUf4Pm .

Solution

Pitest-maven should add all of its dependencies to the forked classpath. This will allow JUnit 5 users following this configuration to declare the dependency on JUnit-Jupiter-Engine as described prior.

~~I will open a PR to remove the dependency filter.~~

~~Alternatively, Pitest-maven could detect the dependencies of Maven surefire. However, that seems error-prone as it creates fragile dependencies between maven plugins, which is not really the Maven way of doing things.~~

[A248]

What maven-surefire seems to do is maintain the possible test providers which have to be added to the forked classpath:

https://github.com/apache/maven-surefire/blob/14bce33c7a350d85df8db49f00cf650556d1131a/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/AbstractSurefireMojo.java#L3288

[hcoles]

Thanks for raising and looking into this. I think cherry picking dependencies surefire style is probably the way to go