error-prone icon indicating copy to clipboard operation
error-prone copied to clipboard
verified publisher icon google

NullPointerException when using matcher in Refaster since 2.16

Open PhilippWendler opened this issue 3 years ago • 2 comments

I have the following Refaster rule that produces a NullPointerException when being applied to one of our projects with Error Prone 2.16, but not with 2.15.0:

package org.sosy_lab.refaster.rules.guava.immutable;

import com.google.common.collect.ImmutableCollection;
import com.google.errorprone.matchers.CompileTimeConstantExpressionMatcher;
import com.google.errorprone.refaster.annotation.AfterTemplate;
import com.google.errorprone.refaster.annotation.BeforeTemplate;
import com.google.errorprone.refaster.annotation.Matches;

class ImmutableCollectionBuilderAddAll<E> {

  @BeforeTemplate
  void addInLoop(
      Iterable<E> source,
      @Matches(CompileTimeConstantExpressionMatcher.class) ImmutableCollection.Builder<E> target) {
    for (E e : source) {
      target.add(e);
    }
  }

  @AfterTemplate
  void addAll(Iterable<E> source, ImmutableCollection.Builder<E> target) {
    target.addAll(source);
  }
}

The compiled version of this rule that I am using is in rule.zip.

I am using OpenJDK 11.0.16 on Ubuntu 22.04, as mentioned with Error Prone 2.16.

The stack trace is

[javac] An exception has occurred in the compiler (11.0.16). Please file a bug against the Java compiler via the Java bug reporting page (http://bugreport.java.com) a
 checking the Bug Database (http://bugs.java.com) for duplicates. Include your program and the following diagnostic in your report. Thank you.
[javac] java.lang.NullPointerException
[javac]     at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:889)
[javac]     at com.google.errorprone.VisitorState.withPath(VisitorState.java:223)
[javac]     at com.google.errorprone.refaster.UMatches.makeVisitorState(UMatches.java:99)
[javac]     at com.google.errorprone.refaster.UMatches.matches(UMatches.java:85)
[javac]     at com.google.errorprone.refaster.UMatches.lambda$defaultAction$0(UMatches.java:60)
[javac]     at com.google.errorprone.refaster.Choice$2.condition(Choice.java:117)
[javac]     at com.google.errorprone.refaster.UMatches.defaultAction(UMatches.java:60)
[javac]     at com.google.errorprone.refaster.UMatches.defaultAction(UMatches.java:36)
[javac]     at jdk.compiler/com.sun.source.util.SimpleTreeVisitor.visitIdentifier(SimpleTreeVisitor.java:587)
[javac]     at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCIdent.accept(JCTree.java:2272)
[javac]     at com.google.errorprone.refaster.UTree.unify(UTree.java:35)
[javac]     at com.google.errorprone.refaster.UTree.unify(UTree.java:31)
[javac]     at com.google.errorprone.refaster.Unifier.unifyNullable(Unifier.java:118)
[javac]     at com.google.errorprone.refaster.Unifier.lambda$unifications$0(Unifier.java:108)
[javac]     at com.google.errorprone.refaster.Choice$2.thenChoose(Choice.java:122)
[javac]     at com.google.errorprone.refaster.UMemberSelect.visitIdentifier(UMemberSelect.java:72)
[javac]     at com.google.errorprone.refaster.UMemberSelect.visitIdentifier(UMemberSelect.java:34)
[javac]     at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCIdent.accept(JCTree.java:2272)
[javac]     at com.google.errorprone.refaster.UTree.unify(UTree.java:35)
[javac]     at com.google.errorprone.refaster.UMethodInvocation.visitMethodInvocation(UMethodInvocation.java:56)
[javac]     at com.google.errorprone.refaster.UMethodInvocation.visitMethodInvocation(UMethodInvocation.java:35)
[javac]     at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1666)
[javac]     at com.google.errorprone.refaster.UTree.unify(UTree.java:35)
[javac]     at com.google.errorprone.refaster.UExpressionStatement.visitExpressionStatement(UExpressionStatement.java:43)
[javac]     at com.google.errorprone.refaster.UExpressionStatement.visitExpressionStatement(UExpressionStatement.java:30)
[javac]     at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCExpressionStatement.accept(JCTree.java:1476)
[javac]     at com.google.errorprone.refaster.UTree.unify(UTree.java:35)
[javac]     at com.google.errorprone.refaster.USimpleStatement.apply(USimpleStatement.java:50)
[javac]     at com.google.errorprone.refaster.USimpleStatement.apply(USimpleStatement.java:30)
[javac]     at com.google.errorprone.refaster.Choice$2.thenChoose(Choice.java:122)
[javac]     at com.google.errorprone.refaster.UBlock.unifyStatementList(UBlock.java:55)
[javac]     at com.google.errorprone.refaster.UBlock.visitBlock(UBlock.java:76)
[javac]     at com.google.errorprone.refaster.UBlock.visitBlock(UBlock.java:35)
[javac]     at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCBlock.accept(JCTree.java:1048)
[javac]     at com.google.errorprone.refaster.UTree.unify(UTree.java:35)
[javac]     at com.google.errorprone.refaster.UTree.unify(UTree.java:31)
[javac]     at com.google.errorprone.refaster.Unifier.unifyNullable(Unifier.java:118)
[javac]     at com.google.errorprone.refaster.Unifier.lambda$unifications$0(Unifier.java:108)
[javac]     at com.google.errorprone.refaster.Choice$2.thenChoose(Choice.java:122)
[javac]     at com.google.errorprone.refaster.UEnhancedForLoop.visitEnhancedForLoop(UEnhancedForLoop.java:72)
[javac]     at com.google.errorprone.refaster.UEnhancedForLoop.visitEnhancedForLoop(UEnhancedForLoop.java:31)
[javac]     at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCEnhancedForLoop.accept(JCTree.java:1188)
[javac]     at com.google.errorprone.refaster.UTree.unify(UTree.java:35)
[javac]     at com.google.errorprone.refaster.USimpleStatement.apply(USimpleStatement.java:50)
[javac]     at com.google.errorprone.refaster.USimpleStatement.apply(USimpleStatement.java:30)
[javac]     at com.google.errorprone.refaster.Choice$2.thenChoose(Choice.java:122)
[javac]     at com.google.errorprone.refaster.BlockTemplate.matchesStartingAtBeginning(BlockTemplate.java:120)
[javac]     at com.google.errorprone.refaster.BlockTemplate.matchesStartingAnywhere(BlockTemplate.java:169)
[javac]     at com.google.errorprone.refaster.BlockTemplate.match(BlockTemplate.java:101)
[javac]     at com.google.errorprone.refaster.RefasterScanner.scan(RefasterScanner.java:107)
[javac]     at com.google.errorprone.refaster.RefasterScanner.scan(RefasterScanner.java:51)
[javac]     at jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:90)
[javac]     at jdk.compiler/com.sun.source.util.TreeScanner.visitMethod(TreeScanner.java:206)
[javac]     at com.google.errorprone.refaster.RefasterScanner.visitMethod(RefasterScanner.java:88)
[javac]     at com.google.errorprone.refaster.RefasterScanner.visitMethod(RefasterScanner.java:51)
[javac]     at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:898)
[javac]     at com.google.errorprone.refaster.RefasterScanner.visitClass(RefasterScanner.java:75)
[javac]     at com.google.errorprone.refaster.RefasterScanner.visitClass(RefasterScanner.java:51)
[javac]     at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:808)
[javac]     at jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:86)
[javac]     at com.google.errorprone.refaster.RefasterScanner.scan(RefasterScanner.java:132)
[javac]     at com.google.errorprone.refaster.RefasterScanner.scan(RefasterScanner.java:51)
[javac]     at jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
[javac]     at jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
[javac]     at jdk.compiler/com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:144)
[javac]     at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:591)
[javac]     at jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:86)
[javac]     at com.google.errorprone.refaster.RefasterScanner.scan(RefasterScanner.java:132)
[javac]     at com.google.errorprone.refaster.RefasterRule.apply(RefasterRule.java:139)
[javac]     at com.google.errorprone.CompositeCodeTransformer.apply(CompositeCodeTransformer.java:45)
[javac]     at com.google.errorprone.ErrorProneAnalyzer.finished(ErrorProneAnalyzer.java:152)
[javac]     at jdk.compiler/com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:132)
[javac]     at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1414)
[javac]     at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1361)
[javac]     at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:960)
[javac]     at jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:311)
[javac]     at jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:170)
[javac]     at jdk.compiler/com.sun.tools.javac.Main.compile(Main.java:57)
[javac]     at jdk.compiler/com.sun.tools.javac.Main.main(Main.java:43)
[javac] Compile failed; see the compiler error output for details.

The crash does not occur if I remove @Matches(CompileTimeConstantExpressionMatcher.class). We have other (working) rules with @Matches, so maybe it is related only to CompileTimeConstantExpressionMatcher?

Unfortunately, it is hard to debug further because I do not know a way how to find out which of our 2800 Java files causes this. If I run the compiler with verbose=true, the last lines before the crash are

...
[javac] [checking org.sosy_lab.cpachecker.util.variableclassification.PartitionTest]                                                                                  
[javac] [checking org.sosy_lab.cpachecker.util.variableclassification.VariableClassificationBuilder]                                                                  
[javac] [loading /home/wendler/git/cpachecker/lib/java/runtime/error_prone_annotations.jar(/com/google/errorprone/annotations/CompileTimeConstant.class)]             
[javac] [loading modules/java.xml/javax/xml/transform/TransformerFactoryConfigurationError.class]                                                                     
[javac] [loading modules/java.xml/javax/xml/transform/TransformerConfigurationException.class]                                                                        
[javac] [total 18070ms]

However, it does not seem as if VariableClassificationBuilder is the problematic class because the crash still occurs if I delete basically all source code from this class except for empty methods. The annotation CompileTimeConstant is not used in our code base (but maybe in libraries).

So if you have some hints on what could I do to find out the problematic source code, I would be happy to debug this further.

Alternatively, here is how this could be reproduced locally:

  • Download and unzip https://github.com/sosy-lab/cpachecker/archive/d805126d8e9562bb5c457cec9d0c280f5be6ca2a.zip
  • sed -e 's/2\.15\.0/2.16/' -i lib/ivy.xml
  • Download and unzip rule.zip.
  • Run with ant refaster -Drefaster.rule.file=rule.refaster (this will download all required build dependencies including Error Prone 2.16).
  • To try out with a different Error Prone JAR, the JARs in lib/java/build/ can be replaced if -Divy.disable=true is passed to ant after the first time (otherwise ant will overwrite the JARs again).

If there is anything else I could do to help debugging this, please let me know.

PhilippWendler avatar Oct 13 '22 08:10 PhilippWendler

Thanks, and sorry. (I added that null check....) I believe @nick-someone has been looking at an internal report we just got about this a couple days ago.

cpovirk avatar Oct 13 '22 10:10 cpovirk

(The above change by @cushon works around this. I don't know if he has plans to go further at this point.)

cpovirk avatar Oct 14 '22 20:10 cpovirk

I don't know if he has plans to go further at this point.

Not at the moment :)

cushon avatar Oct 26 '22 20:10 cushon