cannot ignore code-commenting in javaparser

[feiwww]

When use ‘gumtree jsondiff soucefile1 soucefile2’ to compare two .java source file that only have two diffrent line, got 129 actions and 1208 unmatched node. I guess ignore code-commenting could solve this problem.

my test is based on the commit https://github.com/mockito/mockito/commit/7dd3c17471abd131174f197f6d98cd3fce5a9af5#diff-ddbc0d47e06a589778a6efe24c5d71d3

[jrfaller]

Mmm it seems to be the case, However I am not sure whether or not parsing comments should be turned on. It would may be make sense to let this choice up to the user but it would require rethinking how user can furnish options to the tree generators, but it would be definitely useful.

I will mark that as a feature request :)

Thanks for your feedback!

[s0nata]

@jrfaller and when one is using GTD through the APIs is there a way to get the comments?

As of v.3.0.0-beta2 I was only able to get source code ASTs - if I am missing something please let me know!

[jrfaller]

You mean that code comments are not part of the AST returned by JavaParser?

[s0nata]

@jrfaller seems so - as if all comments were ignored, even though JavaParser is totally able to extract them.

so on my end, if I have this source file of 15 lines, that has a doc, a block and an inline comment:

public class Dummy {

  /**
   * dummy doc comment
   *
   * @param a
   */
  public void foo(int a) {
    /* block comment of useless assignment */
    int b = a - 1;

    // some inline comment
    System.out.println("hello");
  }
}

and if I parse and print like this below:

    Run.initClients();
    Run.initGenerators();

    Tree srcTree;
    try {
      // parse the file into and AST
      srcTree = new JavaParserGenerator().generateFrom().file(srcFile).getRoot();
      // print tree contents
      System.out.println(srcTree.getDescendants());
      System.out.println(srcTree.toTreeString());

    }

there are no comment nodes:

[ClassOrInterfaceDeclaration [0,235], Modifier: public [0,7], SimpleName: Dummy [13,19], MethodDeclaration [78,233], Modifier: public [78,85], SimpleName: foo [90,94], Parameter [94,100], PrimitiveType: int [94,98], SimpleName: a [98,100], VoidType [85,90], BlockStmt [101,233], ExpressionStmt [153,168], VariableDeclarationExpr [153,167], VariableDeclarator [157,167], PrimitiveType: int [153,157], SimpleName: b [157,159], BinaryExpr [161,167], NameExpr [161,163], SimpleName: a [161,163], IntegerLiteralExpr: 1 [165,167], ExpressionStmt [200,229], MethodCallExpr [200,228], FieldAccessExpr [200,211], NameExpr [200,207], SimpleName: System [200,207], SimpleName: out [207,211], SimpleName: println [211,219], StringLiteralExpr: hello [219,227]]

CompilationUnit [0,236]
    ClassOrInterfaceDeclaration [0,235]
        Modifier: public [0,7]
        SimpleName: Dummy [13,19]
        MethodDeclaration [78,233]
            Modifier: public [78,85]
            SimpleName: foo [90,94]
            Parameter [94,100]
                PrimitiveType: int [94,98]
                SimpleName: a [98,100]
            VoidType [85,90]
            BlockStmt [101,233]
                ExpressionStmt [153,168]
                    VariableDeclarationExpr [153,167]
                        VariableDeclarator [157,167]
                            PrimitiveType: int [153,157]
                            SimpleName: b [157,159]
                            BinaryExpr [161,167]
                                NameExpr [161,163]
                                    SimpleName: a [161,163]
                                IntegerLiteralExpr: 1 [165,167]
                ExpressionStmt [200,229]
                    MethodCallExpr [200,228]
                        FieldAccessExpr [200,211]
                            NameExpr [200,207]
                                SimpleName: System [200,207]
                            SimpleName: out [207,211]
                        SimpleName: println [211,219]
                        StringLiteralExpr: hello [219,227]

Process finished with exit code 0

or is it the case that comment nodes are just not printed out by GTD?

[jrfaller]

OK I have made some informal tests and it seems that JavaParser's provided visitor do not visit comment nodes as they are part of a dedicated comment attribute and not in the regular children node list. I think this is doable to make them available in GT's AST. I have hacked a quick intregration, but for the following case:

/**
 * class javadoc comment
 */
public class Foo {
    /**
     * field javadoc comment
     */
    public int foo; // field inline comment

    /**
     * method javadoc comment
     */
    public void bar() {
        return; // method inline comment
    }
}

I get the following AST :

CompilationUnit [0,260]
    ClassOrInterfaceDeclaration [33,260]
        JavadocComment: * class javadoc comment
  [0,32]
        Modifier: public [33,39]
        SimpleName: Foo [46,49]
        FieldDeclaration [101,116]
            LineComment:  field inline comment [117,140]
            Modifier: public [101,107]
            VariableDeclarator [112,115]
                PrimitiveType: int [108,111]
                SimpleName: foo [112,115]
        MethodDeclaration [192,258]
            JavadocComment: * method javadoc comment
      [146,187]
            Modifier: public [192,198]
            SimpleName: bar [204,207]
            VoidType [199,203]
            BlockStmt [210,258]
                ReturnStmt [220,227]
                    LineComment:  method inline comment [228,252]
        JavadocComment: * field javadoc comment
      [56,96]

That seems OK except for the javadoc of the field which is oddly placed. WDYT ?

[monperrus]

FYI https://github.com/SpoonLabs/gumtree-spoon-ast-diff/ supports comments in Java ASTs. (and it is significantly better than the JavaParser backend, but I'm (fully) biased :-)