jazzer icon indicating copy to clipboard operation
jazzer copied to clipboard

Published 20 hours ago • verified publisher icon CodeIntelligenceTesting

when running pitest.org tests on jazz code it gives

Open nagkumar opened this issue 2 years ago • 4 comments 

5:26:59 pm PIT >> FINE : MINION : java.lang.ClassCircularityError: com/code_intelligence/jazzer/runtime/JazzerInternal
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.regex.Matcher.match(Matcher.java:1755)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.regex.Matcher.matches(Matcher.java:712)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.commons.util.ClassNamePatternFilterUtils.lambda$createPredicateFromPatterns$4(ClassNamePatternFilterUtils.java:83)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1602)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.stream.ReferencePipeline.noneMatch(ReferencePipeline.java:642)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.commons.util.ClassNamePatternFilterUtils.lambda$createPredicateFromPatterns$5(ClassNamePatternFilterUtils.java:83)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.function.Predicate.lambda$or$2(Predicate.java:101)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.commons.util.ExceptionUtils.pruneStackTrace(ExceptionUtils.java:126)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.StackTracePruningEngineExecutionListener.pruneStackTrace(StackTracePruningEngineExecutionListener.java:55)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
5:26:59 pm PIT >> FINE : MINION :       at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1092)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.StackTracePruningEngineExecutionListener.executionFinished(StackTracePruningEngineExecutionListener.java:49)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.DelegatingEngineExecutionListener.executionFinished(DelegatingEngineExecutionListener.java:46)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.OutcomeDelayingEngineExecutionListener.reportEngineFailure(OutcomeDelayingEngineExecutionListener.java:83)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:208)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:172)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:95)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:60)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:144)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:59)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47)
5:26:59 pm PIT >> FINE : MINION :       at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:63)
5:26:59 pm PIT >> FINE : MINION :       at org.pitest.junit5.JUnit5TestUnitFinder.findTestUnits(Unknown Source)
5:26:59 pm PIT >> FINE : MINION :       at org.pitest.mutationtest.config.PrioritisingTestUnitFinder.findTestUnits(Unknown Source)
5:26:59 pm PIT >> FINE : MINION :       at org.pitest.testapi.execute.FindTestUnits.findTestUnits(Unknown Source)
5:26:59 pm PIT >> FINE : MINION :       at org.pitest.testapi.execute.FindTestUnits.getTestUnits(Unknown Source)
5:26:59 pm PIT >> FINE : MINION :       at org.pitest.testapi.execute.FindTestUnits.findTestUnitsForAllSuppliedClasses(Unknown Source)
5:26:59 pm PIT >> FINE : MINION :       at org.pitest.coverage.execute.CoverageMinion.discoverTests(Unknown Source)
5:26:59 pm PIT >> FINE : MINION :       at org.pitest.coverage.execute.CoverageMinion.getTestsFromParent(Unknown Source)
5:26:59 pm PIT >> FINE : MINION :       at org.pitest.coverage.execute.CoverageMinion.main(Unknown Source)
5:26:59 pm PIT >> INFO : MINION : 5:26:59 pm PIT >> SEVERE : Error calculating coverage. Process will exit.
5:26:59 pm PIT >> INFO : MINION : java.lang.ClassCircularityError: com/code_intelligence/jazzer/runtime/JazzerInternal
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.regex.Matcher.match(Matcher.java:1755)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.regex.Matcher.matches(Matcher.java:712)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.commons.util.ClassNamePatternFilterUtils.lambda$createPredicateFromPatterns$4(ClassNamePatternFilterUtils.java:83)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1602)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.stream.ReferencePipeline.noneMatch(ReferencePipeline.java:642)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.commons.util.ClassNamePatternFilterUtils.lambda$createPredicateFromPatterns$5(ClassNamePatternFilterUtils.java:83)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.function.Predicate.lambda$or$2(Predicate.java:101)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.commons.util.ExceptionUtils.pruneStackTrace(ExceptionUtils.java:126)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.StackTracePruningEngineExecutionListener.pruneStackTrace(StackTracePruningEngineExecutionListener.java:55)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
5:26:59 pm PIT >> INFO : MINION :       at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1092)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.StackTracePruningEngineExecutionListener.executionFinished(StackTracePruningEngineExecutionListener.java:49)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.DelegatingEngineExecutionListener.executionFinished(DelegatingEngineExecutionListener.java:46)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.OutcomeDelayingEngineExecutionListener.reportEngineFailure(OutcomeDelayingEngineExecutionListener.java:83)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:208)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:172)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:95)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:60)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:144)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:59)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47)
5:26:59 pm PIT >> INFO : MINION :       at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:63)
5:26:59 pm PIT >> INFO : MINION :       at org.pitest.junit5.JUnit5TestUnitFinder.findTestUnits(Unknown Source)
5:26:59 pm PIT >> INFO : MINION :       at org.pitest.mutationtest.config.PrioritisingTestUnitFinder.findTestUnits(Unknown Source)
5:26:59 pm PIT >> INFO : MINION :       at org.pitest.testapi.execute.FindTestUnits.findTestUnits(Unknown Source)
5:26:59 pm PIT >> INFO : MINION :       at org.pitest.testapi.execute.FindTestUnits.getTestUnits(Unknown Source)
5:26:59 pm PIT >> INFO : MINION :       at org.pitest.testapi.execute.FindTestUnits.findTestUnitsForAllSuppliedClasses(Unknown Source)
5:26:59 pm PIT >> INFO : MINION :       at org.pitest.coverage.execute.CoverageMinion.discoverTests(Unknown Source)
5:26:59 pm PIT >> INFO : MINION :       at org.pitest.coverage.execute.CoverageMinion.getTestsFromParent(Unknown Source)
5:26:59 pm PIT >> INFO : MINION :       at org.pitest.coverage.execute.CoverageMinion.main(Unknown Source)
\5:26:59 pm PIT >> SEVERE : Coverage generator Minion exited abnormally due to UNKNOWN_ERROR
Exception in thread "main" org.pitest.util.PitError: Coverage generation minion exited abnormally! (UNKNOWN_ERROR)

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 19.0.2+7-44
Uptime : 16172
Input ->
 1 : -Dfile.encoding=UTF-8
 2 : -Duser.country=IN
 3 : -Duser.language=en
 4 : -Duser.variant
BootClassPathSupported : false


Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 19.0.2+7-44
Uptime : 16172
Input ->
 1 : -Dfile.encoding=UTF-8
 2 : -Duser.country=IN
 3 : -Duser.language=en
 4 : -Duser.variant
BootClassPathSupported : false

        at org.pitest.util.Unchecked.translateCheckedException(Unchecked.java:20)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:106)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:52)
        at org.pitest.mutationtest.tooling.MutationCoverage.runAnalysis(MutationCoverage.java:148)
        at org.pitest.mutationtest.tooling.MutationCoverage.runReport(MutationCoverage.java:138)
        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:129)
        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:57)
        at org.pitest.mutationtest.commandline.MutationCoverageReport.runReport(MutationCoverageReport.java:98)
        at org.pitest.mutationtest.commandline.MutationCoverageReport.main(MutationCoverageReport.java:45)
Caused by: org.pitest.util.PitError: Coverage generation minion exited abnormally! (UNKNOWN_ERROR)

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 19.0.2+7-44
Uptime : 16172
Input ->
 1 : -Dfile.encoding=UTF-8
 2 : -Duser.country=IN
 3 : -Duser.language=en
 4 : -Duser.variant
BootClassPathSupported : false

        at org.pitest.coverage.execute.DefaultCoverageGenerator.gatherCoverageData(DefaultCoverageGenerator.java:148)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:90)
        ... 7 more

nagkumar avatar Jun 04 '23 11:06 nagkumar

@nagkumar Could you test this with 0.17.1? If this still reproduces, could you share a reproducing example?

fmeum avatar Jun 05 '23 20:06 fmeum

I still see this error even with the latest 0.17.1

10:04:25 am PIT >> FINE : MINION : java.lang.ClassCircularityError: com/code_intelligence/jazzer/runtime/JazzerInternal
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.regex.Matcher.match(Matcher.java:1755)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.regex.Matcher.matches(Matcher.java:712)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.commons.util.ClassNamePatternFilterUtils.lambda$createPredicateFromPatterns$4(ClassNamePatternFilterUtils.java:83)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1602)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.stream.ReferencePipeline.noneMatch(ReferencePipeline.java:642)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.commons.util.ClassNamePatternFilterUtils.lambda$createPredicateFromPatterns$5(ClassNamePatternFilterUtils.java:83)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.function.Predicate.lambda$or$2(Predicate.java:101)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.commons.util.ExceptionUtils.pruneStackTrace(ExceptionUtils.java:126)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.StackTracePruningEngineExecutionListener.pruneStackTrace(StackTracePruningEngineExecutionListener.java:55)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
10:04:25 am PIT >> FINE : MINION :      at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1092)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.StackTracePruningEngineExecutionListener.executionFinished(StackTracePruningEngineExecutionListener.java:49)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.DelegatingEngineExecutionListener.executionFinished(DelegatingEngineExecutionListener.java:46)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.OutcomeDelayingEngineExecutionListener.reportEngineFailure(OutcomeDelayingEngineExecutionListener.java:83)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:208)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:172)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:95)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:60)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:144)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:59)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47)
10:04:25 am PIT >> FINE : MINION :      at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:63)
10:04:25 am PIT >> FINE : MINION :      at org.pitest.junit5.JUnit5TestUnitFinder.findTestUnits(Unknown Source)
10:04:25 am PIT >> FINE : MINION :      at org.pitest.mutationtest.config.PrioritisingTestUnitFinder.findTestUnits(Unknown Source)
10:04:25 am PIT >> FINE : MINION :      at org.pitest.testapi.execute.FindTestUnits.findTestUnits(Unknown Source)
10:04:25 am PIT >> FINE : MINION :      at org.pitest.testapi.execute.FindTestUnits.getTestUnits(Unknown Source)
10:04:25 am PIT >> FINE : MINION :      at org.pitest.testapi.execute.FindTestUnits.findTestUnitsForAllSuppliedClasses(Unknown Source)
10:04:25 am PIT >> FINE : MINION :      at org.pitest.coverage.execute.CoverageMinion.discoverTests(Unknown Source)
10:04:25 am PIT >> FINE : MINION :      at org.pitest.coverage.execute.CoverageMinion.getTestsFromParent(Unknown Source)
10:04:25 am PIT >> FINE : MINION :      at org.pitest.coverage.execute.CoverageMinion.main(Unknown Source)
10:04:25 am PIT >> INFO : MINION : 10:04:25 am PIT >> SEVERE : Error calculating coverage. Process will exit.

I am not sure how to isolate this.. except that it is coming in from pitest.org mutation testing time.

nagkumar avatar Jun 09 '23 04:06 nagkumar

As far as I understand, pitest modifies the bytecode under test at runtime - just like Jazzer. It is very unlikely that this would "just work" and likely requires a pretty good understanding of both Jazzer and pitest to resolve, if at all possible.

While combining bytecode mutation testing and fuzzing sounds very interesting, it's not something we will likely get to look into in the foreseeable future. If anyone wants to dig deeper on this and understands the root cause of the incompatibility better, please post here.

fmeum avatar Jul 05 '23 14:07 fmeum

@nagkumar - same as I wrote on your other issue, ping me and we can discuss other potential options or workarounds.

ghost avatar Feb 26 '24 12:02 ghost