jazzer
jazzer copied to clipboard
Published
20 hours ago •
CodeIntelligenceTesting
CodeIntelligenceTesting
junit tests don't generate java code to reproduce exceptions
uname -a
Linux archlinux 6.1.5-arch2-1 #1 SMP PREEMPT_DYNAMIC Thu, 12 Jan 2023 22:42:33 +0000 x86_64 GNU/Linux
mvn -version
Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: /opt/maven
Java version: 19.0.1, vendor: N/A, runtime: /usr/lib/jvm/java-19-openjdk
Default locale: ru_RU, platform encoding: UTF-8
OS name: "linux", version: "6.1.5-arch2-1", arch: "amd64", family: "unix"
mvn test
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------------< freedom1b2830:test >-------------------------
[INFO] Building test 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ test ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/user_dev_new/eclipse-workspace/test/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ test ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/user_dev_new/eclipse-workspace/test/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M7:test (default-test) @ test ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.example.AutofuzzFuzzTest
INFO: Loaded 159 hooks from com.code_intelligence.jazzer.runtime.TraceCmpHooks
INFO: Loaded 4 hooks from com.code_intelligence.jazzer.runtime.TraceDivHooks
INFO: Loaded 2 hooks from com.code_intelligence.jazzer.runtime.TraceIndirHooks
INFO: Loaded 4 hooks from com.code_intelligence.jazzer.runtime.NativeLibHooks
INFO: Loaded 5 hooks from com.code_intelligence.jazzer.sanitizers.Deserialization
INFO: Loaded 5 hooks from com.code_intelligence.jazzer.sanitizers.ExpressionLanguageInjection
INFO: Loaded 70 hooks from com.code_intelligence.jazzer.sanitizers.LdapInjection
INFO: Loaded 46 hooks from com.code_intelligence.jazzer.sanitizers.NamingContextLookup
INFO: Loaded 1 hooks from com.code_intelligence.jazzer.sanitizers.OsCommandInjection
INFO: Loaded 48 hooks from com.code_intelligence.jazzer.sanitizers.ReflectiveCall
INFO: Loaded 8 hooks from com.code_intelligence.jazzer.sanitizers.RegexInjection
INFO: Loaded 16 hooks from com.code_intelligence.jazzer.sanitizers.RegexRoadblocks
INFO: Loaded 19 hooks from com.code_intelligence.jazzer.sanitizers.SqlInjection
INFO: Instrumented com.example.AutofuzzFuzzTest (took 186 ms, size +11%)
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3546565980
INFO: Loaded 1 modules (512 inline 8-bit counters): 512 [0x7fab04931f30, 0x7fab04932130),
INFO: Loaded 1 PC tables (512 PCs): 512 [0x7fab048b0b70,0x7fab048b2b70),
INFO: 5 files found in /home/user_dev_new/eclipse-workspace/test/.cifuzz-corpus/com.example.AutofuzzFuzzTest
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: seed corpus: files: 5 min: 1b max: 10b total: 27b rss: 674Mb
INFO: Instrumented com.example.ExploreMe (took 38 ms, size +111%)
#6 INITED cov: 15 ft: 15 corp: 5/27b exec/s: 0 rss: 674Mb
#32768 pulse cov: 15 ft: 15 corp: 5/27b lim: 331 exec/s: 16384 rss: 674Mb
#65536 pulse cov: 15 ft: 15 corp: 5/27b lim: 659 exec/s: 21845 rss: 674Mb
MS: 2 CMP-EraseBytes- DE: "jaz.Zer"-; base unit: 872ba8e6cdb8fd1598d67a39d2f93875fa39407a
0x40,0x6a,0x61,0x7a,0x2e,0x5a,0x65,0x72,0x69,0x73,0xa,0x58,0x58,0x58,0xce,0x58,
@jaz.Zeris\012XXX\316X
artifact_prefix='/home/user_dev_new/eclipse-workspace/test/'; Test unit written to /home/user_dev_new/eclipse-workspace/test/crash-b6d7d4e892cc5af63b34f111bc62c8a4a26325b8
Base64: QGphei5aZXJpcwpYWFjOWA==
Done 118343 runs in 4 second(s)
[ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 23.201 s <<< FAILURE! - in com.example.AutofuzzFuzzTest
[ERROR] com.example.AutofuzzFuzzTest.myFuzzTest(FuzzedDataProvider)[1] Time elapsed: 5.461 s <<< ERROR!
com.code_intelligence.jazzer.api.FuzzerSecurityIssueHigh:
Remote Code Execution
Unrestricted class loading based on externally controlled data may allow
remote code execution depending on available classes on the classpath.
at jaz.Zer.<clinit>(Zer.java:54)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:390)
at java.base/java.lang.Class.forName(Class.java:381)
at com.example.ExploreMe.exploreMe(ExploreMe.java:13)
at com.example.AutofuzzFuzzTest.myFuzzTest(AutofuzzFuzzTest.java:21)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] AutofuzzFuzzTest.myFuzzTest:21 » FuzzerSecurityIssueHigh Remote Code Execution
Unrestricted class loading based on externally controlled data may allow
remote code execution depending on available classes on the classpath.
[INFO]
[ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 1
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 36.589 s
[INFO] Finished at: 2023-01-20T09:39:33Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test (default-test) on project test:
[ERROR]
[ERROR] Please refer to /home/user_dev_new/eclipse-workspace/test/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
ls -la
drwxr-xr-x. 1 user_dev_new user_dev_new 234 янв 20 09:39 .
drwxr-xr-x. 1 user_dev_new user_dev_new 526 янв 5 16:34 ..
drwxr-xr-x. 1 user_dev_new user_dev_new 56 дек 31 11:33 .cifuzz-corpus
-rw-r--r--. 1 user_dev_new user_dev_new 1478 дек 28 10:09 .classpath
-rw-r--r--. 1 user_dev_new user_dev_new 16 янв 20 09:39 crash-b6d7d4e892cc5af63b34f111bc62c8a4a26325b8
drwxr-xr-x. 1 user_dev_new user_dev_new 144 янв 2 12:10 .git
-rw-r--r--. 1 user_dev_new user_dev_new 39 дек 31 10:46 .gitignore
-rw-r--r--. 1 user_dev_new user_dev_new 1443 янв 20 09:38 pom.xml
-rw-r--r--. 1 user_dev_new user_dev_new 533 дек 28 08:07 .project
drwxr-xr-x. 1 user_dev_new user_dev_new 168 дек 28 10:09 .settings
drwxr-xr-x. 1 user_dev_new user_dev_new 16 дек 28 08:07 src
drwxr-xr-x. 1 user_dev_new user_dev_new 172 янв 20 09:39 target
find src/ -type f
src/main/java/com/example/ExploreMe.java
src/test/java/com/example/AutofuzzFuzzTest.java
pom.xml
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
<version>3.0.2</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.9.2</version>
</dependency>
<dependency>
<groupId>com.code-intelligence</groupId>
<artifactId>jazzer-junit</artifactId>
<version>0.14.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M7</version>
</plugin>
</plugins>
<testResources>
<testResource>
<directory>${project.basedir}/src/test/resources</directory>
</testResource>
</testResources>
</build>
Correct, this is currently not supported by the JUnit integration.
Findings are stored in the corpus directory and used in subsequent regression mode runs, which is a nicer way to reproduce/debug findings to begin with. Have you tried that out?
How to use the JUnit integration is described at Using Jazzer via JUnit 5, but you saw that as you're using the integration already.
Point 5. describes how to use found issues in the regression test mode. In this mode you can start the test like a normal JUnit test from within your IDE, set breakpoints and basically do everything you could do in a normal unit test. You could also only execute the test of a specific input to easily reproduce the found issue. How to do that is IDE dependent, though.
Eclipse IDE for Java Developers (includes Incubating components) Version: 2022-12 (4.26.0) Build id: 20221201-1913
without JAZZER_FUZZ=1
1)mvn clean install
2)eclipse run test
java.lang.IllegalStateException: Failed to run Agent.install
at com.code_intelligence.jazzer.agent.AgentInstaller.install(AgentInstaller.java:49)
at com.code_intelligence.jazzer.junit.FuzzTestArgumentsProvider.configureAndInstallAgent(FuzzTestArgumentsProvider.java:71)
at com.code_intelligence.jazzer.junit.FuzzTestArgumentsProvider.provideArguments(FuzzTestArgumentsProvider.java:80)
at org.junit.jupiter.params.ParameterizedTestExtension.arguments(ParameterizedTestExtension.java:145)
at org.junit.jupiter.params.ParameterizedTestExtension.lambda$provideTestTemplateInvocationContexts$2(ParameterizedTestExtension.java:90)
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:269)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:110)
at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:44)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:95)
at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:91)
at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:60)
at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.run(JUnit5TestReference.java:98)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:40)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:529)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
Suppressed: org.junit.platform.commons.PreconditionViolationException: Configuration error: You must configure at least one set of arguments for this @ParameterizedTest
at org.junit.platform.commons.util.Preconditions.condition(Preconditions.java:299)
at org.junit.jupiter.params.ParameterizedTestExtension.lambda$provideTestTemplateInvocationContexts$5(ParameterizedTestExtension.java:98)
at java.util.stream.AbstractPipeline.close(AbstractPipeline.java:323)
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:279)
... 56 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.code_intelligence.jazzer.agent.AgentInstaller.install(AgentInstaller.java:46)
... 72 more
Caused by: java.lang.VerifyError
at sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
at sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:144)
at com.code_intelligence.jazzer.agent.Agent.installInternal(Agent.kt:148)
at com.code_intelligence.jazzer.agent.Agent.installInternal$default(Agent.kt:36)
at com.code_intelligence.jazzer.agent.Agent.install(Agent.kt:33)
... 77 more
This should be worked around quite effectively by https://github.com/CodeIntelligenceTesting/jazzer/commit/48ff37c56954ca50c4439f7023006026c22c4057, even though I can't tell what the root cause is.
@freedom1b2830 Please test with the next Jazzer release (not out yet).
