Pitest cannot run the coverage test suite with JUnit 5.9.1 parameterized tests with method source

[davidburstrom]

trafficstars

Something has changed internally between JUnit 5.9.0 and 5.9.1, because now all tests that use @org.junit.jupiter.params.ParameterizedTest with @org.junit.jupiter.params.provider.MethodSource are failing. The verbose logging doesn't print any failure message, but I can see that the indicated method doesn't even get invoked.

I don't have anything public that reproduces this issue at this point, sorry.

Using Pitest 1.9.5 with pitest JUnit plugin version 1.0.0.

[davidburstrom]

After investigating this and https://junit.org/junit5/docs/5.9.1/release-notes/ I've found that one way to fix this is to release a new version of the pitest-junit5-plugin where the junit.platform.version is bumped to 1.9.1, so that methods required by JUnit Jupiter 5.9.1 is available.

The problem is that MethodArgumentsProvider requires org.junit.platform.commons.util.CollectionUtils.isConvertibleToStream (https://github.com/junit-team/junit5/commit/d1f6edee15ccacfbb931c5f98a2eacc04ede5f56#diff-69531322f458eb134daa846fd4239d5d1a5874bea76c426b6ca0f2a6708c06f1R92) which is only available in Platform 1.9.1 (https://github.com/junit-team/junit5/commit/8f9460f2bad44bd74ebbb957d125735af7e9ee38)

JUnit Platform seems to be sufficiently backwards compatible so that it can be bumped on its own in pitest-junit5-plugin.

[hcoles]

Thanks @davidburstrom,

I'll try and take a proper look at this tomorrow. On the face of it the platform should be at provided scope so this sort of issue wouldn't happen. I have a feeling there was a reason why this couldn't be done though.

[davidburstrom]

I'm creating a PR as we speak :)

[davidburstrom]

I'll try and take a proper look at this tomorrow. On the face of it the platform should be at provided scope so this sort of issue wouldn't happen. I have a feeling there was a reason why this couldn't be done though.

Yes, agreed, I don't quite see why the JUnit libraries should be shaded. I vaguely remember seeing issues with this before, where the shaded versions weren't compatible with other JUnit dependencies on the classpath. It should be easy enough for them to be provided, and instead add some detection in the plugin that can verify that the versions are within bounds, or adding compatibility layers.

[Allsimon]

It looks like the bug is still present if you omit the MethodSource value

For example, PITest will not instrument void someTests(TestCase testCase) { here:

    @MethodSource
    @ParameterizedTest
    void someTests(TestCase testCase) {
        // ...
    }

    static List<TestCase> someTests() {

but if we specify manually the method name it will work:

    @MethodSource("someTests")
    @ParameterizedTest
    void someTests(TestCase testCase) {
        // ...
    }

    static List<TestCase> someTests() {

[hcoles]

@Allsimon Could you open a new issue and provide a minimal project which reproduces this issue for you. From a quick play parameterized tests without and explicit method source seem to work correctly.

[MiklosKiss82]

With jUnit 5.9.2, the bug appears in a lot of cases again.

[hcoles]

@MiklosKiss82 Something looks to have changed internally in junit platform between 1.9.1 and 1.9.2. Releasing a build (1.1.2) against 1.9.2 now, which may solve the issue. If not, can you create a minimal project that reproduces the problem.

[MiklosKiss82]

@MiklosKiss82 Something looks to have changed internally in junit platform between 1.9.1 and 1.9.2. Releasing a build (1.1.2) against 1.9.2 now, which may solve the issue. If not, can you create a minimal project that reproduces the problem.

Thank you very much. At first sight, it does solve the problem.