Bug: java.lang.NoSuchMethodError

[danglotb]

Hello,

I'm trying to run Flacoco on this project but I have the following error:

Exception in thread "main" java.lang.NoSuchMethodError: 'org.junit.platform.engine.ConfigurationParameters org.junit.platform.launcher.TestPlan.getConfigurationParameters()'
	at org.junit.platform.launcher.listeners.UniqueIdTrackingListener.testPlanExecutionStarted(UniqueIdTrackingListener.java:143)
	at org.junit.platform.launcher.core.TestExecutionListenerRegistry$CompositeTestExecutionListener.lambda$testPlanExecutionStarted$6(TestExecutionListenerRegistry.java:97)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at org.junit.platform.launcher.core.TestExecutionListenerRegistry.notifyTestExecutionListeners(TestExecutionListenerRegistry.java:59)
	at org.junit.platform.launcher.core.TestExecutionListenerRegistry.access$100(TestExecutionListenerRegistry.java:28)
	at org.junit.platform.launcher.core.TestExecutionListenerRegistry$CompositeTestExecutionListener.testPlanExecutionStarted(TestExecutionListenerRegistry.java:97)
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:183)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:202)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:181)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
	at eu.stamp_project.testrunner.runner.JUnit5Runner.run(JUnit5Runner.java:98)
	at eu.stamp_project.testrunner.runner.coverage.JUnit5OnlineJacocoRunner.executeCoveredTestPerTestMethod(JUnit5OnlineJacocoRunner.java:30)
	at eu.stamp_project.testrunner.runner.coverage.JacocoOnlineRunner.runCoveredTestResultPerTestMethod(JacocoOnlineRunner.java:41)
	at eu.stamp_project.testrunner.runner.coverage.JUnit5OnlineJacocoRunner.main(JUnit5OnlineJacocoRunner.java:48)

Here the toString() of the used FlacocoConfig:

FlacocoConfig{workspace='/home/benjamin/workspace/diff-jjoules-demo/.', projectPath='/home/benjamin/workspace/diff-jjoules-demo', srcJavaDir=[/home/benjamin/workspace/diff-jjoules-demo/src/main/java], srcTestDir=[/home/benjamin/workspace/diff-jjoules-demo/src/test], binJavaDir=[/home/benjamin/workspace/diff-jjoules-demo/target/classes], binTestDir=[/home/benjamin/workspace/diff-jjoules-demo/target/test-classes], classpath='/home/benjamin/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.8.2/junit-jupiter-api-5.8.2.jar:/home/benjamin/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar:/home/benjamin/.m2/repository/org/junit/platform/junit-platform-commons/1.8.2/junit-platform-commons-1.8.2.jar:/home/benjamin/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar:/home/benjamin/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.8.2/junit-jupiter-engine-5.8.2.jar:/home/benjamin/.m2/repository/org/junit/platform/junit-platform-engine/1.8.2/junit-platform-engine-1.8.2.jar:/home/benjamin/.m2/repository/org/junit/platform/junit-platform-launcher/1.8.2/junit-platform-launcher-1.8.2.jar:/home/benjamin/.m2/repository/junit/junit/4.13.2/junit-4.13.2.jar:/home/benjamin/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar', customJUnitClasspath='null',  customJacocoClasspath='null',  mavenHome='/home/benjamin/.m2/repository/',  coverTests=false, testRunnerVerbose=true,  testRunnerTimeoutInMs=1000000,  testRunnerJVMArgs='null',  threshold=0.0,  includeZeros=false, complianceLevel=8,  testDetectionStrategy=CLASSLOADER,  ignoredTests=[],  jUnit4Tests=[],  jUnit5Tests=[], jacocoIncludes=[], jacocoExcludes=[], family=SPECTRUM_BASED, spectrumFormula=OCHIAI, computeSpoonResults=false}
[0] INFO Flacoco - Running Flacoco...
Parsing --sourceBinaries /home/benjamin/workspace/diff-jjoules-demo/target/classes --testBinaries /home/benjamin/workspace/diff-jjoules-demo/target/test-classes --class fr.davidson.diff_jjoules_demo.TestCode   --coverage-detail DETAIL_COMPRESSED

Seems that there is a conflict between the versions of org.junit.platform.launcher where the method getConfigurationParameters() of ConfigurationParameters has been introduced in 1.8 while Flacoco is running in 1.3:

@API(status=MAINTAINED,
     since="1.8")
public ConfigurationParameters getConfigurationParameters()

Get the ConfigurationParameters for this test plan.
Returns:
    the configuration parameters; never null
Since:
    1.8

source

However, I did try to bump the versions in Flacoco (and in STAMP-project/test-runner) but it did not work :disappointed:

Have you any idea?

Thank you very much!

[monperrus]

Seems that there is a conflict between the versions of org.junit.platform.launcher

fully agree. do we use the latest version in flacoco and test runner?

[danglotb]

Hello @monperrus

No, we do not.

#130 attempts to do so for Flacoco but we do not have renovate for STAMP-project/test-runner.

However, I am afraid that it will not be sufficient since I tried locally to do this and it did not resolve the issue.

Maybe I am missing something :dizzy_face:

[monperrus]

The one which is taken is the one that happens the last in the pom.xml of flacoco.

If junit is after test-runner, the junit version of flacoco is taken

If junit is before test-runner, the junit version of test-runner is taken.

Same thing in your client project.

[sophiakolak]

Encountering the same error on defects4j Jsoup bugs. Did this ever get solved?

[andre15silva]

Hi @sophiakolak,

We never got to fix this one, no.

Could you let me know in which Jsoup bugs you have encountered this issue, and the command you are using to run flacoco?

[sophiakolak]

Hi @andre15silva, the ones that are not working are the 1 line bugs for Jsoup & Mockito (listed below)

Jsoup [46, 9, 51, 47, 88, 17, 37, 61, 77, 57, 41, 15, 62, 34, 43, 25, 45, 86, 32]
Mockito [5, 26, 8, 34, 38, 24, 29]

It works in most cases, but for some reason these two are throwing an error that looks very similar to the one above. The command I'm running is:

command = f"java -jar ~/flacoco/target/flacoco-1.0.6-SNAPSHOT-jar-with-dependencies.jar -p {self.temp_dir} " + \
            f" --workspace={self.temp_dir}" + \
            f" --srcTestDir={self.temp_dir}/{test_dir} --binTestDir={self.temp_dir}/{test_bin}" + \ 
            f"--srcJavaDir={self.temp_dir}/{src_dir} --binJavaDir={self.temp_dir}/{src_bin}" + \
            f" -v -o {out_path} --format CSV " 

So, a sample command

java -jar ~/flacoco/target/flacoco-1.0.6-SNAPSHOT-jar-with-dependencies.jar -p /Users/sdkolak/d4j_temp/Mockito/5_b  --workspace=/Users/sdkolak/d4j_temp/Mockito/5_b --srcTestDir=/Users/sdkolak/d4j_temp/Mockito/5_b/test --binTestDir=/Users/sdkolak/d4j_temp/Mockito/5_b/build/classes/test --srcJavaDir=/Users/sdkolak/d4j_temp/Mockito/5_b/src --binJavaDir=/Users/sdkolak/d4j_temp/Mockito/5_b/build/classes/main -v -o /Users/sdkolak/entropy_fl/sbfl/Mockito_5.csv --format CSV 

[andre15silva]

Hi @sophiakolak,

I believe https://github.com/STAMP-project/test-runner/pull/146 fixes the bug you encountered when running flacoco on the projects you mentioned. With that patch, I was able to run flacoco on all the bugs you mentioned. As soon as the PR on test-runner is approved and merged, I will update flacoco and release 1.0.6 with the fix included.

Furthermore, you will also want to provide the project's classpath to flacoco in your command. On Defects4J, you can obtain this by running the command defects4j export -p cp.test on the project you wish to inspect. Keep in mind that there is a bug with the defects4j export command (https://github.com/rjust/defects4j/issues/39), so you will want to compile the project after exporting the classpath.

PS: the bug I fixed is different from the one initially reported by Benjamin, which remains unsolved, but I believe it does not affect you in these cases.

[andre15silva]

1.0.6 has been released :) @sophiakolak

[sophiakolak]

awesome :) thank you!