[SPIKE] [spring-boot-ls] Investigate replacing JDT with OpenRewrite Java

[BoykoAlex]

Given the age and support of Eclipse JDT project as well as its somewhat "ducttapy" integration into Spring Boot LS (reflection in CU cache) it might be a good idea to replace it with something lighter, modern and well supported such as rewrite-java from Open Rewrite.

Work in progress is in rewrite-java branch: https://github.com/spring-projects/sts4/tree/rewrite-java

The good so far:

• the project is on the rise
• should be more performant
• lighter and cleaner api
• opens doors for supporting similar languages implemented currently (Groovy) as well as in the future (Kotlin?) by Open Rewrite

The bad so far:

• very poor support for java sources with incomplete AST branches (i.e. private String prop = breaks parsing)
• cannot resolve compile time variable values (i.e. static constant values)
• time consuming to fully replace JDT with rewrite-java
• larger JAR footprint

(cc: @martinlippert @kdvolder @nierajsingh - just an FYI that this work is in progress)

[martinlippert]

@BoykoAlex Feel free to push work-in-progress into a branch for this, would be nice to see some of the changes that would be necessary in code.

Few thoughts here:

• we would need to make the incomplete AST situations work across the place. My assumption here is that code completion, content-assist, snippets, etc., they should all work for code that has broken syntax - as this will most likely be the case when developers work on the code in the editor. We need to avoid that our support breaks frequently (and from the users point of view for unknown reasons) due to broken syntax.
• what is the JAR footprint of rewrite-java? (compared to JDT)
• how quickly is rewrite-java adopting new Java versions? We need to be able to quickly ship support for newer JDKs (beyond LTS versions), so would be good to know what the support policy here is

[BoykoAlex]

@martinlippert

Added the branch in the issue description section. Duplicating here as well: https://github.com/spring-projects/sts4/tree/rewrite-java

Incomplete AST cases to be raised or perhaps even fixed as I work on this.

JAR footprint:

• JDT: 37Mb
• rewrite-java: 60Mb

Support for new java versions - TBD.

[BoykoAlex]

Was looking at the java parser performance today. It is very important to have the following message on the execution context when using the java parser for our purposes:

ctx.putMessage(JavaParser.SKIP_SOURCE_SET_TYPE_GENERATION, true);

This avoids parsing more sources and cache their types it seems greatly improving parsing performance.

So far it looks like the first parse with OR parser takes ~200 ms and 8-12 ms subsequent. While with JDT miraculously takes 20 ms for the first and about the same 6-12 ms for subsequent (???!!!). Not sure how we managed to achieve this with JDT. Hope we're not missing anything with JDT parsing...

[martinlippert]

For the symbol generation we parse Java code using JDT in two phases: we parse Java source code without diving into method bodies in a first pass. Only if we find anything that requires method bodies to be parsed as well, we re-parse that specific source file again with method bodies included (e.g. when we extract symbols for WebFlux routes).

What happens when using open-rewrite? Does it generate an AST for everything or can you trim it down to skip method bodies as well? (might improve parsing performance).

[martinlippert]

With the work for https://github.com/spring-projects/sts4/issues/1068, we are essentially moving back out of using OpenRewrite for validation purposes and continue to use OpenRewrite purely for applying code changes (like small quick fixes as well as large version upgrades).

Therefore, closing this one as done.