rewrite
rewrite copied to clipboard
Java 17 Parser Support
- [x] Support
instanceof
pattern matching - [ ] Support multi-line strings
- [ ] Support switch expressions
- [ ] Support records
I’m not sure if this belongs in this repo or the main openrewrite repo but… what are the plans for support of java 17 record syntax and all the upcoming pattern matching features please?
Thank you - and thanks for the amazing tool!
Our biggest technical hurdle is the constraints Jigsaw is placing on us. We either need to map from ECJ or find a way around the inability to open internal modules like the compiler in Java 17.
Do you mean so that openrewrite can run under jdk17? That’s already working for me - the —add-opens lines that you so helpfully document made this just work. I actually meant parsing the jdk17 syntax, understanding records and so forth. Let me know if I’m misunderstanding your answer please - I don’t mean to waste your time! And if you want me to specifically give examples of what doesn’t work for me that’s obviously fair enough.
Here is a specific usecase that's breaking me:
The class file below produces the stack trace included at the bottom.
======
package rewrite;
import com.gs.ctl.eventgraph.IdentityRef;
public class C {
// Works
private class RawEdgeC {
IdentityRef idr;
}
// Works
private record RawEdgeI(int i) {
}
// Works
private record RawEdgeO(Object i) {
}
// Bang
private record RawEdgeBoom(IdentityRef id) {
}
}
=====
org.openrewrite.java.JavaParsingException: Failed to convert for the following cursor stack:--- BEGIN PATH ---
JCCompilationUnit(sourceFile = /home/coder/IdeaProjects/ctl-core/development-tools/rewrite/smoke-test-class.java)
JCClassDecl(name = C, line = 6)
JCClassDecl(name = RawEdgeBoom, line = 23)
--- END PATH ---
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1483)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1489)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1598)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1580)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:461)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:860)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1460)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1489)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1598)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1580)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:461)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:860)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1460)
at org.openrewrite.java.Java11ParserVisitor.convertAll(Java11ParserVisitor.java:1513)
at org.openrewrite.java.Java11ParserVisitor.visitCompilationUnit(Java11ParserVisitor.java:503)
at org.openrewrite.java.Java11ParserVisitor.visitCompilationUnit(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:614)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11Parser.lambda$parseInputs$0(Java11Parser.java:175)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at org.openrewrite.java.Java11Parser.parseInputs(Java11Parser.java:197)
at org.openrewrite.Parser.parse(Parser.java:39)
at rewrite.MigrateToFluentLoggerRecipe.main(MigrateToFluentLoggerRecipe.java:168)
Caused by: java.lang.IndexOutOfBoundsException: Illegal start index
at java.base/java.util.regex.Matcher.find(Matcher.java:770)
at org.openrewrite.java.Java11ParserVisitor.lambda$visitVariables$28(Java11ParserVisitor.java:1369)
at org.openrewrite.java.Java11ParserVisitor.visitVariables(Java11ParserVisitor.java:1376)
at org.openrewrite.java.Java11ParserVisitor.visitVariable(Java11ParserVisitor.java:1324)
at org.openrewrite.java.Java11ParserVisitor.visitVariable(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCVariableDecl.accept(JCTree.java:1045)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1460)
... 33 more
org.openrewrite.java.JavaParsingException: Failed to convert for the following cursor stack:--- BEGIN PATH ---
JCCompilationUnit(sourceFile = /home/coder/IdeaProjects/ctl-core/development-tools/rewrite/smoke-test-class.java)
JCClassDecl(name = C, line = 6)
--- END PATH ---
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1483)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1489)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1598)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1580)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:461)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:860)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1460)
at org.openrewrite.java.Java11ParserVisitor.convertAll(Java11ParserVisitor.java:1513)
at org.openrewrite.java.Java11ParserVisitor.visitCompilationUnit(Java11ParserVisitor.java:503)
at org.openrewrite.java.Java11ParserVisitor.visitCompilationUnit(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:614)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11Parser.lambda$parseInputs$0(Java11Parser.java:175)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at org.openrewrite.java.Java11Parser.parseInputs(Java11Parser.java:197)
at org.openrewrite.Parser.parse(Parser.java:39)
at rewrite.MigrateToFluentLoggerRecipe.main(MigrateToFluentLoggerRecipe.java:168)
Caused by: java.lang.IndexOutOfBoundsException: Illegal start index
at java.base/java.util.regex.Matcher.find(Matcher.java:770)
at org.openrewrite.java.Java11ParserVisitor.lambda$visitVariables$28(Java11ParserVisitor.java:1369)
at org.openrewrite.java.Java11ParserVisitor.visitVariables(Java11ParserVisitor.java:1376)
at org.openrewrite.java.Java11ParserVisitor.visitVariable(Java11ParserVisitor.java:1324)
at org.openrewrite.java.Java11ParserVisitor.visitVariable(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCVariableDecl.accept(JCTree.java:1045)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1460)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1489)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1598)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1580)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:461)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:860)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1460)
... 25 more
org.openrewrite.java.JavaParsingException: Failed to convert for the following cursor stack:--- BEGIN PATH ---
JCCompilationUnit(sourceFile = /home/coder/IdeaProjects/ctl-core/development-tools/rewrite/smoke-test-class.java)
--- END PATH ---
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1483)
at org.openrewrite.java.Java11ParserVisitor.convertAll(Java11ParserVisitor.java:1513)
at org.openrewrite.java.Java11ParserVisitor.visitCompilationUnit(Java11ParserVisitor.java:503)
at org.openrewrite.java.Java11ParserVisitor.visitCompilationUnit(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:614)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11Parser.lambda$parseInputs$0(Java11Parser.java:175)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at org.openrewrite.java.Java11Parser.parseInputs(Java11Parser.java:197)
at org.openrewrite.Parser.parse(Parser.java:39)
at rewrite.MigrateToFluentLoggerRecipe.main(MigrateToFluentLoggerRecipe.java:168)
Caused by: java.lang.IndexOutOfBoundsException: Illegal start index
at java.base/java.util.regex.Matcher.find(Matcher.java:770)
at org.openrewrite.java.Java11ParserVisitor.lambda$visitVariables$28(Java11ParserVisitor.java:1369)
at org.openrewrite.java.Java11ParserVisitor.visitVariables(Java11ParserVisitor.java:1376)
at org.openrewrite.java.Java11ParserVisitor.visitVariable(Java11ParserVisitor.java:1324)
at org.openrewrite.java.Java11ParserVisitor.visitVariable(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCVariableDecl.accept(JCTree.java:1045)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1460)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1489)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1598)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1580)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:461)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:860)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1460)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1489)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1598)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1580)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:461)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:860)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1460)
... 17 more
java.lang.IndexOutOfBoundsException: Illegal start index
at java.base/java.util.regex.Matcher.find(Matcher.java:770)
at org.openrewrite.java.Java11ParserVisitor.lambda$visitVariables$28(Java11ParserVisitor.java:1369)
at org.openrewrite.java.Java11ParserVisitor.visitVariables(Java11ParserVisitor.java:1376)
at org.openrewrite.java.Java11ParserVisitor.visitVariable(Java11ParserVisitor.java:1324)
at org.openrewrite.java.Java11ParserVisitor.visitVariable(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCVariableDecl.accept(JCTree.java:1045)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1460)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1489)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1598)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1580)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:461)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:860)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1460)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1489)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1598)
at org.openrewrite.java.Java11ParserVisitor.convertStatements(Java11ParserVisitor.java:1580)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:461)
at org.openrewrite.java.Java11ParserVisitor.visitClass(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:860)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11ParserVisitor.convert(Java11ParserVisitor.java:1460)
at org.openrewrite.java.Java11ParserVisitor.convertAll(Java11ParserVisitor.java:1513)
at org.openrewrite.java.Java11ParserVisitor.visitCompilationUnit(Java11ParserVisitor.java:503)
at org.openrewrite.java.Java11ParserVisitor.visitCompilationUnit(Java11ParserVisitor.java:69)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:614)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at org.openrewrite.java.Java11Parser.lambda$parseInputs$0(Java11Parser.java:175)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at org.openrewrite.java.Java11Parser.parseInputs(Java11Parser.java:197)
at org.openrewrite.Parser.parse(Parser.java:39)
Any thoughts/plans on this one please?
Thanks!
As an interim, would it be hard to ensure that OpenRewrite always fails when encountering constructs it doesn't understand? At the moment OR will sometimes successfully parse, but the result when actually passed through a recipe is gibberish. I'm trying to find a minimal reproducible case but haven't got one yet.
Turns out a minimal reproducible example case is just ‘anything with a variable-declaring instanceof expression’. So, OR processing the below expression will result in gibberish. Note that this isn’t the only thing - sometimes record syntax makes it sad as well.
if( foo instanceof String s && s.isEmpty() ) { …. }
Hi @Jonpez2 , thanks for all the information! I just want to add a comment so that anyone who is watching this issue knows we haven't forgotten about it. I think Java 17 support will be one of @tkvangorder 's next big projects.
Well that’s good news thank you.
Is there any way to get the pulse on the support for java 17+ please? Thank you!
Hi @Jonpez2, @tkvangorder has looked into the Jigsaw constraints and has a promising solution. Development of Java 17 support is in process.
Superb! That includes the syntactic support as well as the runtime support, correct?
I ask because it’s the syntax changes in jdk 17 that are killing my use of OR, not the jigsaw runtime restrictions
Also looks like we might get this bit of awesome in jdk19 - https://openjdk.java.net/jeps/405. Java’s getting hard to keep up with! :)
@Jonpez2 Yes, the plan is first to get the parser working without being constrained by jigsaw and then to implement the additional language features.
Ok cool. I can’t wait!
I see you released jdk 17 runtime support. Awesome to see movement, thank you!
Yep, we are halfway there, we can load the parser in jigsaw without security exceptions, the next step is adding 17 language features. Hoping to work on that soon, it has been a busy couple of weeks
Super news. Thank you so much for the hard work!
On Sun, 12 Jun 2022 at 21:17, Tyler Van Gorder @.***> wrote:
Yep, we are halfway there, we can load the parser in jigsaw without security exceptions, the next step is adding 17 language features. Hoping to work on that soon, it has been a busy couple of weeks
— Reply to this email directly, view it on GitHub https://github.com/openrewrite/rewrite/issues/1501#issuecomment-1153283095, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABN425ICFAQL43SMCGMMTULVOZAWZANCNFSM5RRTR3RA . You are receiving this because you were mentioned.Message ID: @.***>
Hello! How is this coming along please? Thank you!
Hi, @Jonpez2 This is still on our TODO list, thanks for your patience.
OK. I wonder if you should do all the way through to jdk19, since that's coming in like 2 months...
I knojw it's a lot to ask, but what kind of timeframe should I have in mind here please? Just wondering whether to find myself some kind of a stopgap for the interim...
Thank you!
Ping? Thanks!
Superb! That includes the syntactic support as well as the runtime support, correct?
On Fri, 13 May 2022 at 15:22, Tracey Yoshima @.***> wrote:
Hi @Jonpez2 https://github.com/Jonpez2, @tkvangorder https://github.com/tkvangorder has looked into the Jigsaw constraints and has a promising solution. Development of Java 17 support is in process.
— Reply to this email directly, view it on GitHub https://github.com/openrewrite/rewrite/issues/1501#issuecomment-1126372445, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABN425OOMQNJRSKLPORY5RTVJ2TZBANCNFSM5RRTR3RA . You are receiving this because you were mentioned.Message ID: @.***>
That's correct @Jonpez2. Just finished up switch expressions on my trip home. Records are next!
Yay! thank you!
Ooh exciting thank you!!
On Fri, 14 Oct 2022 at 05:22, Jonathan Schnéider @.***> wrote:
Closed #1501 https://github.com/openrewrite/rewrite/issues/1501 as completed.
— Reply to this email directly, view it on GitHub https://github.com/openrewrite/rewrite/issues/1501#event-7586849667, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABN425PMRAF6N7RCQHXD2FTWDDN2BANCNFSM5RRTR3RA . You are receiving this because you were mentioned.Message ID: @.***>