jbehave-junit-runner icon indicating copy to clipboard operation
jbehave-junit-runner copied to clipboard

Published 20 hours ago • verified publisher icon valfirst

Using JUnitReportingRunner with pitest

Open danielleberre opened this issue 6 years ago • 2 comments

I use with my students JUnit 4.12, JBehave 4.3.5 and pitest 1.4.5.

Without using JUnitReportingRunner, I can run pitest on my test suite (with both JUnit and JBehave test cases).

When using JUnitReportingRunner, I get the following error when pitest is enabled:

[pitest] 15:01:12 PIT >> INFO : MINION : 15:01:12 PIT >> INFO : Checking environment
   [pitest] 
   [pitest] 15:01:12 PIT >> INFO : MINION : 15:01:12 PIT >> INFO : Found  86 tests
   [pitest] 
   [pitest] 15:01:12 PIT >> INFO : MINION : 15:01:12 PIT >> INFO : Dependency analysis reduced number of potential tests by 0
   [pitest] 
   [pitest] 15:01:12 PIT >> INFO : MINION : 15:01:12 PIT >> INFO : 86 tests received
   [pitest] 
   [pitest] 15:01:12 PIT >> INFO : MINION : 15:01:12 PIT >> WARNING : JUnit error for class class java.lang.Object : java.lang.Object
   [pitest] 
   [pitest] 15:01:12 PIT >> INFO : MINION : 15:01:12 PIT >> WARNING : Not able to filter java.lang.Object. Mutation may have prevented JUnit from constructing test
   [pitest] 
   [pitest] 15:01:12 PIT >> INFO : MINION : 15:01:12 PIT >> SEVERE : Description [testClass=java.lang.Object, name=AfterStories(java.lang.Object)]
   [pitest] java.lang.Exception: No runnable methods
   [pitest] 	at org.junit.runners.BlockJUnit4ClassRunner.validateInstanceMethods(BlockJUnit4ClassRunner.java:191)
   [pitest] 	at org.ju
   [pitest] 15:01:12 PIT >> INFO : MINION : nit.runners.BlockJUnit4ClassRunner.collectInitializationErrors(BlockJUnit4ClassRunner.java:128)
   [pitest] 	at org.junit.runners.ParentRunner.validate(ParentRunner.java:416)
   [pitest] 	at org.junit.runners.ParentRunner.<init>(ParentRunner.java:84)
   [pitest] 	at org.junit.runners.BlockJU
   [pitest] 15:01:12 PIT >> INFO : MINION : nit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:65)
   [pitest] 	at org.junit.internal.builders.JUnit4Builder.runnerForClass(JUnit4Builder.java:10)
   [pitest] 	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
   [pitest] 	at org.junit.internal.builders.A
   [pitest] 15:01:12 PIT >> INFO : MINION : llDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
   [pitest] 	at org.pitest.junit.adapter.AdaptedJUnitTestUnit.createRunner(AdaptedJUnitTestUnit.java:107)
   [pitest] 	at org.pitest.junit.adapter.AdaptedJUnitTestUnit.execute(AdaptedJUnitTestUni
   [pitest] 15:01:12 PIT >> INFO : MINION : t.java:62)
   [pitest] 	at org.pitest.coverage.execute.CoverageDecorator.execute(CoverageDecorator.java:50)
   [pitest] 	at org.pitest.testapi.execute.containers.UnContainer.execute(UnContainer.java:31)
   [pitest] 	at org.pitest.testapi.execute.Pitest.executeTests(Pitest.java:57)
   [pitest] 	at org.pi
   [pitest] 15:01:12 PIT >> INFO : MINION : test.testapi.execute.Pitest.run(Pitest.java:48)
   [pitest] 	at org.pitest.coverage.execute.CoverageWorker.run(CoverageWorker.java:51)
   [pitest] 	at org.pitest.coverage.execute.CoverageMinion.main(CoverageMinion.java:90)
   [pitest] 15:01:12 PIT >> WARNING : JUnit error for class class jav
   [pitest] 15:01:12 PIT >> INFO : MINION : a.lang.Object : java.lang.Object
   [pitest] 15:01:12 PIT >> WARNING : Not able to filter java.lang.Object. Mutation may have prevented JUnit from constructing test
   [pitest] 
   [pitest] 15:01:12 PIT >> INFO : MINION : 15:01:12 PIT >> SEVERE : Description [testClass=java.lang.Object, name=BeforeStories(java.lang.Object)]
   [pitest] java.lang.Exception: No runnable methods
   [pitest] 	at org.junit.runners.BlockJUnit4ClassRunner.validateInstanceMethods(BlockJUnit4ClassRunner.java:191)
   [pitest] 	at org.j
   [pitest] 15:01:12 PIT >> INFO : MINION : unit.runners.BlockJUnit4ClassRunner.collectInitializationErrors(BlockJUnit4ClassRunner.java:128)
   [pitest] 	at org.junit.runners.ParentRunner.validate(ParentRunner.java:416)
   [pitest] 	at org.junit.runners.ParentRunner.<init>(ParentRunner.java:84)
   [pitest] 	at org.junit.runners.BlockJ
   [pitest] 15:01:12 PIT >> INFO : MINION : Unit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:65)
   [pitest] 	at org.junit.internal.builders.JUnit4Builder.runnerForClass(JUnit4Builder.java:10)
   [pitest] 	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
   [pitest] 	at org.junit.internal.builders.
   [pitest] 15:01:12 PIT >> INFO : MINION : AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
   [pitest] 	at org.pitest.junit.adapter.AdaptedJUnitTestUnit.createRunner(AdaptedJUnitTestUnit.java:107)
   [pitest] 	at org.pitest.junit.adapter.AdaptedJUnitTestUnit.execute(AdaptedJUnitTestUn
   [pitest] 15:01:12 PIT >> INFO : MINION : it.java:62)
   [pitest] 	at org.pitest.coverage.execute.CoverageDecorator.execute(CoverageDecorator.java:50)
   [pitest] 	at org.pitest.testapi.execute.containers.UnContainer.execute(UnContainer.java:31)
   [pitest] 	at org.pitest.testapi.execute.Pitest.executeTests(Pitest.java:57)
   [pitest] 	at org.p
   [pitest] 15:01:12 PIT >> INFO : MINION : itest.testapi.execute.Pitest.run(Pitest.java:48)
   [pitest] 	at org.pitest.coverage.execute.CoverageWorker.run(CoverageWorker.java:51)
   [pitest] 	at org.pitest.coverage.execute.CoverageMinion.main(CoverageMinion.java:90)
   [pitest] 
   [pitest] 15:01:13 PIT >> SEVERE : Description [testClass=java.lang.Object, name=AfterStories(java.lang.Object)] did not pass without mutation.
   [pitest] 15:01:13 PIT >> SEVERE : Description [testClass=java.lang.Object, name=BeforeStories(java.lang.Object)] did not pass without mutation.
   [pitest] 15:01:13 PIT >> INFO : MINION : 15:01:13 PIT >> WARNING : More threads at end of test (6) testEfficiencyOfPrependHidden(migl.util.ConsListHiddenTest) than start. (5)
   [pitest] 
   [pitest] 15:01:13 PIT >> INFO : Calculated coverage in 1 seconds.
   [pitest] 15:01:13 PIT >> SEVERE : Tests failing without mutation: 
   [pitest] Description [testClass=java.lang.Object, name=AfterStories(java.lang.Object)]
   [pitest] Description [testClass=java.lang.Object, name=BeforeStories(java.lang.Object)]
   [pitest] Exception in thread "main" org.pitest.help.PitHelpError: 2 tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite.
   [pitest] See http://pitest.org for more details.
   [pitest] 	at org.pitest.coverage.execute.DefaultCoverageGenerator.verifyBuildSuitableForMutationTesting(DefaultCoverageGenerator.java:114)
   [pitest] 	at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:96)
   [pitest] 	at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:51)
   [pitest] 	at org.pitest.mutationtest.tooling.MutationCoverage.runReport(MutationCoverage.java:115)
   [pitest] 	at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:120)
   [pitest] 	at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:50)
   [pitest] 	at org.pitest.mutationtest.commandline.MutationCoverageReport.runReport(MutationCoverageReport.java:87)
   [pitest] 	at org.pitest.mutationtest.commandline.MutationCoverageReport.main(MutationCoverageReport.java:45)
   [pitest]-

I would like to continue using JUnitReportingRunner because it returns properly within eclipse or ant the number of tests that pass, fail or abort in JBehave.

Any hints of what could cause that issue?

danielleberre avatar Feb 16 '19 14:02 danielleberre

@danielleberre could you please provide a small project to reproduce the issue?

valfirst avatar Feb 18 '19 11:02 valfirst

I just invited you to a private project "jbehavejunitreportingrunnerpbm" containing the code and tests allowing to reproduce the issue.

With ant, just launch

ant m1

danielleberre avatar Feb 18 '19 14:02 danielleberre