joern
joern copied to clipboard
joernio
[Bug] Error executing query
Describe the bug To Reproduce Steps to reproduce the behavior:
- run cmd
./joern-scan ./tests/code/javasrc/ --overwrite --language java - sometimes see error:
[INFO ] initializing 337 nodes from existing storage
[INFO ] installed GC monitors. will clear references if heap (after GC) is larger than 80%
[INFO ] Start of pass: io.joern.x2cpg.passes.base.FileCreationPass
[INFO ] Pass io.joern.x2cpg.passes.base.FileCreationPass completed in 10 ms (25% on mutations). 26 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.joern.x2cpg.passes.base.NamespaceCreator
[INFO ] Pass io.joern.x2cpg.passes.base.NamespaceCreator completed in 3 ms (26% on mutations). 7 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.joern.x2cpg.passes.base.TypeDeclStubCreator
[INFO ] Pass io.joern.x2cpg.passes.base.TypeDeclStubCreator completed in 5 ms (13% on mutations). 24 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.joern.x2cpg.passes.base.MethodStubCreator
[INFO ] Pass io.joern.x2cpg.passes.base.MethodStubCreator completed in 16 ms (17% on mutations). 163 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.joern.x2cpg.passes.base.ParameterIndexCompatPass
[INFO ] Pass io.joern.x2cpg.passes.base.ParameterIndexCompatPass completed in 1 ms (31% on mutations). 39 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.joern.x2cpg.passes.base.MethodDecoratorPass
[INFO ] Pass io.joern.x2cpg.passes.base.MethodDecoratorPass completed in 4 ms (41% on mutations). 192 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.joern.x2cpg.passes.base.AstLinkerPass
[INFO ] Pass io.joern.x2cpg.passes.base.AstLinkerPass completed in 6 ms (3% on mutations). 41 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of enhancement: io.joern.x2cpg.passes.base.ContainsEdgePass
[INFO ] Enhancement io.joern.x2cpg.passes.base.ContainsEdgePass completed in 23 ms. 159 + 0 changes committed from 66 parts.
[INFO ] Start of pass: io.joern.x2cpg.passes.base.TypeUsagePass
[INFO ] Pass io.joern.x2cpg.passes.base.TypeUsagePass completed in 7 ms (15% on mutations). 314 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.shiftleft.semanticcpg.Overlays$$anon$1
[INFO ] Pass io.shiftleft.semanticcpg.Overlays$$anon$1 completed in 1 ms (10% on mutations). 1 + 0 changes committed from 1 parts.
[INFO ] Start of enhancement: io.joern.x2cpg.passes.controlflow.CfgCreationPass
[INFO ] Enhancement io.joern.x2cpg.passes.controlflow.CfgCreationPass completed in 35 ms. 139 + 0 changes committed from 34 parts.
[INFO ] Start of pass: io.joern.x2cpg.passes.controlflow.cfgdominator.CfgDominatorPass
[INFO ] Pass io.joern.x2cpg.passes.controlflow.cfgdominator.CfgDominatorPass completed in 24 ms (2% on mutations). 276 + 0 changes committed from 34 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.joern.x2cpg.passes.controlflow.codepencegraph.CdgPass
[INFO ] Pass io.joern.x2cpg.passes.controlflow.codepencegraph.CdgPass completed in 8 ms (1% on mutations). 3 + 0 changes committed from 34 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.shiftleft.semanticcpg.Overlays$$anon$1
[INFO ] Pass io.shiftleft.semanticcpg.Overlays$$anon$1 completed in 0 ms (54% on mutations). 1 + 0 changes committed from 1 parts.
[INFO ] Start of pass: io.joern.x2cpg.passes.typerelations.TypeHierarchyPass
[INFO ] Pass io.joern.x2cpg.passes.typerelations.TypeHierarchyPass completed in 3 ms (1% on mutations). 4 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.joern.x2cpg.passes.typerelations.AliasLinkerPass
[INFO ] Pass io.joern.x2cpg.passes.typerelations.AliasLinkerPass completed in 1 ms (1% on mutations). 0 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.shiftleft.semanticcpg.Overlays$$anon$1
[INFO ] Pass io.shiftleft.semanticcpg.Overlays$$anon$1 completed in 0 ms (57% on mutations). 1 + 0 changes committed from 1 parts.
[INFO ] Start of pass: io.joern.x2cpg.passes.callgraph.MethodRefLinker
[INFO ] Pass io.joern.x2cpg.passes.callgraph.MethodRefLinker completed in 1 ms (3% on mutations). 1 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.joern.x2cpg.passes.callgraph.StaticCallLinker
[INFO ] Pass io.joern.x2cpg.passes.callgraph.StaticCallLinker completed in 2 ms (4% on mutations). 18 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.joern.x2cpg.passes.callgraph.DynamicCallLinker
[INFO ] Pass io.joern.x2cpg.passes.callgraph.DynamicCallLinker completed in 20 ms (0% on mutations). 17 + 0 changes committed from 1 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.shiftleft.semanticcpg.Overlays$$anon$1
[INFO ] Pass io.shiftleft.semanticcpg.Overlays$$anon$1 completed in 0 ms (44% on mutations). 1 + 0 changes committed from 1 parts.
[INFO ] Start of pass: io.joern.dataflowengineoss.passes.reachingdef.ReachingDefPass
[INFO ] Calculating reaching definitions for: com.sec.servlet.InfoServlet.UserInfoFound:<unresolvedSignature>(2) in xss.java
[INFO ] Calculating reaching definitions for: io.shiftleft.java2cpg.testcode.methods.MethodTest.annotatedFunction:int(int) in MethodTest.java
[INFO ] Calculating reaching definitions for: javax.servlet.http.HttpServletRequest.getParameter:<unresolvedSignature>(1) in <empty>
[INFO ] Calculating reaching definitions for: java.sql.Connection.createStatement:java.sql.Statement() in <empty>
[INFO ] Calculating reaching definitions for: io.shiftleft.java2cpg.testcode.methods.MethodTest.nonStaticMethod:int(int) in MethodTest.java
[INFO ] Calculating reaching definitions for: java.io.IOException.printStackTrace:void() in <empty>
[INFO ] Calculating reaching definitions for: java.util.ArrayList.<init>:void() in <empty>
[INFO ] Calculating reaching definitions for: java.lang.Object.toString:java.lang.String() in <empty>
[INFO ] Calculating reaching definitions for: com.sec.servlet.InfoServlet.UserInfoFound:<unresolvedSignature>(2) in sql.java
[INFO ] Calculating reaching definitions for: java.lang.Class.forName:<unresolvedSignature>(1) in <empty>
[INFO ] Calculating reaching definitions for: io.shiftleft.java2cpg.testcode.methods.MethodTest.internalMethod:java.lang.String() in MethodTest.java
[INFO ] Calculating reaching definitions for: java.sql.Statement.executeUpdate:int(java.lang.String) in <empty>
[INFO ] Calculating reaching definitions for: javax.servlet.http.HttpServletResponse.getWriter:<unresolvedSignature>(0) in <empty>
[INFO ] Calculating reaching definitions for: com.sec.servlet.InfoServlet.service:<unresolvedSignature>(2) in sql.java
[INFO ] Calculating reaching definitions for: <unresolvedNamespace>.println:<unresolvedSignature>(1) in <empty>
[INFO ] Calculating reaching definitions for: io.shiftleft.java2cpg.testcode.methods.MethodTest.<init>:void() in MethodTest.java
[INFO ] Number of definitions for java.lang.Object.toString:java.lang.String(): 1
[INFO ] Number of definitions for java.sql.Statement.executeUpdate:int(java.lang.String): 3
[INFO ] Number of definitions for <unresolvedNamespace>.println:<unresolvedSignature>(1): 3
[INFO ] Number of definitions for io.shiftleft.java2cpg.testcode.methods.MethodTest.<init>:void(): 1
[INFO ] Number of definitions for javax.servlet.http.HttpServletRequest.getParameter:<unresolvedSignature>(1): 3
[INFO ] Number of definitions for io.shiftleft.java2cpg.testcode.methods.MethodTest.internalMethod:java.lang.String(): 1
[INFO ] Number of definitions for java.util.ArrayList.<init>:void(): 1
[INFO ] Number of definitions for io.shiftleft.java2cpg.testcode.methods.MethodTest.annotatedFunction:int(int): 2
[INFO ] Number of definitions for java.lang.Class.forName:<unresolvedSignature>(1): 3
[INFO ] Number of definitions for java.io.IOException.printStackTrace:void(): 2
[INFO ] Number of definitions for io.shiftleft.java2cpg.testcode.methods.MethodTest.nonStaticMethod:int(int): 2
[INFO ] Number of definitions for javax.servlet.http.HttpServletResponse.getWriter:<unresolvedSignature>(0): 2
[INFO ] Number of definitions for java.sql.Connection.createStatement:java.sql.Statement(): 2
[INFO ] Number of definitions for com.sec.servlet.InfoServlet.service:<unresolvedSignature>(2): 11
[INFO ] Number of definitions for com.sec.servlet.InfoServlet.UserInfoFound:<unresolvedSignature>(2): 14
[INFO ] Number of definitions for com.sec.servlet.InfoServlet.UserInfoFound:<unresolvedSignature>(2): 32
[INFO ] Calculating reaching definitions for: java.util.stream.Stream.map:java.util.stream.Stream(java.util.function.Function) in <empty>
[INFO ] Calculating reaching definitions for: java.sql.DriverManager.getConnection:java.sql.Connection(java.lang.String,java.lang.String,java.lang.String) in <empty>
[INFO ] Calculating reaching definitions for: io.shiftleft.java2cpg.testcode.methods.MethodTest.callsExternalMethod:java.lang.String() in MethodTest.java
[INFO ] Calculating reaching definitions for: javax.servlet.http.HttpServletRequest.setCharacterEncoding:<unresolvedSignature>(1) in <empty>
[INFO ] Calculating reaching definitions for: javax.servlet.http.HttpServletResponse.setContentType:<unresolvedSignature>(1) in <empty>
[INFO ] Calculating reaching definitions for: <operator>.addition in <empty>
[INFO ] Calculating reaching definitions for: io.shiftleft.java2cpg.testcode.methods.MethodTest.callsNoExternalMethod:java.lang.String() in MethodTest.java
[INFO ] Calculating reaching definitions for: java.util.List.stream:java.util.stream.Stream() in <empty>
[INFO ] Number of definitions for java.util.stream.Stream.map:java.util.stream.Stream(java.util.function.Function): 3
[INFO ] Number of definitions for <operator>.addition: 2
[INFO ] Number of definitions for io.shiftleft.java2cpg.testcode.methods.MethodTest.callsExternalMethod:java.lang.String(): 3
[INFO ] Calculating reaching definitions for: <operator>.alloc in <empty>
[INFO ] Number of definitions for java.util.List.stream:java.util.stream.Stream(): 2
[INFO ] Calculating reaching definitions for: <operator>.assignment in <empty>
[INFO ] Number of definitions for io.shiftleft.java2cpg.testcode.methods.MethodTest.callsNoExternalMethod:java.lang.String(): 3
[INFO ] Calculating reaching definitions for: io.shiftleft.java2cpg.testcode.methods.MethodTest.lambda$0:java.lang.Object(java.lang.Object) in MethodTest.java
[INFO ] Number of definitions for javax.servlet.http.HttpServletRequest.setCharacterEncoding:<unresolvedSignature>(1): 3
[INFO ] Number of definitions for java.sql.DriverManager.getConnection:java.sql.Connection(java.lang.String,java.lang.String,java.lang.String): 4
[INFO ] Calculating reaching definitions for: com.sec.servlet.InfoServlet.service:<unresolvedSignature>(2) in xss.java
[INFO ] Calculating reaching definitions for: com.sec.servlet.InfoServlet.<clinit>:void() in <empty>
[INFO ] Calculating reaching definitions for: com.sec.servlet.InfoServlet.<init>:void() in sql.java
[INFO ] Number of definitions for <operator>.alloc: 0
[INFO ] Number of definitions for javax.servlet.http.HttpServletResponse.setContentType:<unresolvedSignature>(1): 3
[INFO ] Number of definitions for io.shiftleft.java2cpg.testcode.methods.MethodTest.lambda$0:java.lang.Object(java.lang.Object): 4
[INFO ] Calculating reaching definitions for: io.shiftleft.java2cpg.testcode.methods.MethodTest.staticMethod:int(int) in MethodTest.java
[INFO ] Number of definitions for com.sec.servlet.InfoServlet.<init>:void(): 1
[INFO ] Number of definitions for com.sec.servlet.InfoServlet.<clinit>:void(): 1
[INFO ] Calculating reaching definitions for: io.shiftleft.java2cpg.testcode.methods.MethodTest.lambdaUser:void() in MethodTest.java
[INFO ] Calculating reaching definitions for: com.sec.servlet.InfoServlet.<clinit>:void() in <empty>
[INFO ] Calculating reaching definitions for: com.sec.servlet.InfoServlet.<init>:void() in xss.java
[INFO ] Number of definitions for com.sec.servlet.InfoServlet.service:<unresolvedSignature>(2): 11
[INFO ] Number of definitions for <operator>.assignment: 2
[INFO ] Number of definitions for com.sec.servlet.InfoServlet.<clinit>:void(): 1
[INFO ] Number of definitions for com.sec.servlet.InfoServlet.<init>:void(): 1
[INFO ] Number of definitions for io.shiftleft.java2cpg.testcode.methods.MethodTest.lambdaUser:void(): 13
[INFO ] Number of definitions for io.shiftleft.java2cpg.testcode.methods.MethodTest.staticMethod:int(int): 1
[INFO ] Pass io.joern.dataflowengineoss.passes.reachingdef.ReachingDefPass completed in 111 ms (2% on mutations). 439 + 0 changes committed from 34 parts. Inverse serialized and stored.
[INFO ] Start of pass: io.shiftleft.semanticcpg.Overlays$$anon$1
[INFO ] Pass io.shiftleft.semanticcpg.Overlays$$anon$1 completed in 0 ms (53% on mutations). 1 + 0 changes committed from 1 parts.
[INFO ] shutdown: start
[INFO ] serializing and persisting 522 nodes (this may take a while)
[INFO ] finished serializing and persisting 522 nodes
[INFO ] shutdown finished
[INFO ] initializing 522 nodes from existing storage
[INFO ] installed GC monitors. will clear references if heap (after GC) is larger than 80%
[INFO ] Start of enhancement: io.joern.console.scan.ScanPass
[WARN ] Error executing query
java.lang.NullPointerException: Cannot invoke "java.lang.CharSequence.length()" because "this.text" is null
at java.util.regex.Matcher.getTextLength(Matcher.java:1769) ~[?:?]
at java.util.regex.Matcher.reset(Matcher.java:415) ~[?:?]
at java.util.regex.Matcher.<init>(Matcher.java:252) ~[?:?]
at java.util.regex.Pattern.matcher(Pattern.java:1134) ~[?:?]
at scala.util.matching.Regex.matches(Regex.scala:476) ~[org.scala-lang.scala-library-2.13.10.jar:?]
at overflowdb.traversal.filter.StringPropertyFilter$.regexp$$anonfun$1(StringPropertyFilter.scala:11) ~[io.shiftleft.overflowdb-traversal_3-1.181.jar:1.181]
at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:479) ~[org.scala-lang.scala-library-2.13.10.jar:?]
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:601) ~[org.scala-lang.scala-library-2.13.10.jar:?]
at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:477) ~[org.scala-lang.scala-library-2.13.10.jar:?]
at scala.collection.Iterator$$anon$9.hasNext(Iterator.scala:583) ~[org.scala-lang.scala-library-2.13.10.jar:?]
at scala.collection.immutable.List.prependedAll(List.scala:152) ~[org.scala-lang.scala-library-2.13.10.jar:?]
at scala.collection.IterableOnceOps.toList(IterableOnce.scala:1288) ~[org.scala-lang.scala-library-2.13.10.jar:?]
at scala.collection.IterableOnceOps.toList$(IterableOnce.scala:1288) ~[org.scala-lang.scala-library-2.13.10.jar:?]
at scala.collection.AbstractIterator.toList(Iterator.scala:1300) ~[org.scala-lang.scala-library-2.13.10.jar:?]
at overflowdb.traversal.TraversalSugarExt$.l$extension(Traversal.scala:24) ~[io.shiftleft.overflowdb-traversal_3-1.181.jar:1.181]
at io.joern.console.scan.package$QueryWrapper.apply(package.scala:36) ~[io.joern.console-20220816.2+6-6c23a69a+20231023-1123.jar:20220816.2+6-6c23a69a+20231023-1123]
at io.joern.console.scan.ScanPass.runOnPart(ScanPass.scala:14) ~[io.joern.console-20220816.2+6-6c23a69a+20231023-1123.jar:20220816.2+6-6c23a69a+20231023-1123]
at io.joern.console.scan.ScanPass.runOnPart(ScanPass.scala:12) ~[io.joern.console-20220816.2+6-6c23a69a+20231023-1123.jar:20220816.2+6-6c23a69a+20231023-1123]
at io.shiftleft.passes.ConcurrentWriterCpgPass.createApplySerializeAndStore$$anonfun$1(ParallelCpgPass.scala:95) ~[io.shiftleft.codepropertygraph_3-1.4.24.jar:1.4.24]
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:678) ~[org.scala-lang.scala-library-2.13.10.jar:?]
at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467) [org.scala-lang.scala-library-2.13.10.jar:?]
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395) [?:?]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) [?:?]
at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) [?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) [?:?]
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) [?:?]
[INFO ] Attempting to determine flows from empty list of sources.
[INFO ] Attempting to determine flows from empty list of sources.
[INFO ] Attempting to determine flows from empty list of sources.
[INFO ] Attempting to determine flows to empty list of sinks.
[INFO ] Attempting to determine flows to empty list of sinks.
[INFO ] Attempting to determine flows to empty list of sinks.
[INFO ] Attempting to determine flows from empty list of sources.
[INFO ] Attempting to determine flows to empty list of sinks.
[INFO ] Attempting to determine flows to empty list of sinks.
[INFO ] Enhancement io.joern.console.scan.ScanPass completed in 85 ms. 0 + 0 changes committed from 24 parts.
[INFO ] Start of pass: io.shiftleft.semanticcpg.Overlays$$anon$1
[INFO ] Pass io.shiftleft.semanticcpg.Overlays$$anon$1 completed in 0 ms (44% on mutations). 1 + 0 changes committed from 1 parts.
Run `joern --for-input-path ./tests/code/javasrc/` to explore interactively
Expected behavior A clear and concise description of what you expected to happen.
Desktop (please complete the following information):
- OS centos 7.9
- Joern Version commit id: 55c51179490823638087e5614f6775b9c75e57ec 2023/10/20
- Java version openjdk version "17.0.7" 2023-04-21 LTS OpenJDK Runtime Environment TencentKonaJDK (build 17.0.7+1-LTS) OpenJDK 64-Bit Server VM TencentKonaJDK (build 17.0.7+1-LTS, mixed mode, sharing)
Additional context Add any other context about the problem here.
