openrewrite
Include DataTable for each Find*** recipe
What problem are you trying to solve?
As the best way today to find about java annotation, method, import, pom dependency, properties, etc during the analysis phase is to use a Find**** recipe able to find a resource in a project/application parsed by openrewrite as LSTree and to look to the csv files generated by the maven goal dry-run, could it be possible to generalize the mechanism able for such recipes to export the result as a DataTable
Describe the solution you'd like
Implement for each Find**** recipe a DataTable class with information to be exported as:
FileName | Resource type | Match ID | Position | Recipe | ...
File path and name of the resource where the condition matched | java or pom or gradle or property or xml or json or yaml or | Match ID = ID passed as parameter from the tool generating the match condition and used to reconcile more easily | If possible include the position: line and chars where the match took place | Recipe where the "find" matched the condition
Have you considered any alternatives or workarounds?
Yes using a marker but it is really hard to scan the generated files post dry-run execution to find files with marker and match what we will find with the initial match query
That would be great to launch openrewrite as a server in order to query it using some Rest services to find and get result directly as we con do with a LanguageServer without the need to read CSV generated files !
Are you interested in contributing this feature to OpenRewrite?
Yes
Thanks for the offer to help @cmoulliard ; Indeed data tables were only added well after we had our find recipes, and weren't backfilled to each of those. Probably best to add these where you see most value first, in individual PRs to make them easier to review and merge.
I agree with you that we will work on PR's cases. Until now I'm looking to propose a MatchingReport DataTable & Row classes that we could use for the different searches
// Row class
public static class Row {
@Column(displayName = "Match ID",
description = "ID of the matching tool used to reconcile the information.")
String matchId;
@Column(displayName = "File's type",
description = "Type of the file where we look for.")
Type type;
@Column(displayName = "Symbol searched",
description = "Symbol about what we search about: annotation, import, method, filed, etc.")
Symbol symbol;
@Column(displayName = "A symbol pattern",
description = "A symbol pattern, expressed as a \"method\" pattern.")
String pattern;
@Column(displayName = "Source file path",
description = "Path of the source file where a match found")
String sourceFilePath;
@Column(displayName = "FQName of the Class",
description = "FQName of the Class containing the symbol we search.")
String className;
}
public enum Type {
JAVA,
POM,
XML,
JSON,
YAML,
PROPERTIES
}
public enum Symbol {
ANNOTATION,
METHOD,
FIELD,
CLASS
}
// Rendered as
"Match ID","File's type","Symbol searched","A symbol pattern","Source file path","FQName of the Class"
"ID of the matching tool used to reconcile the information.","Type of the file where we look for.","Symbol about what we search about: annotation, import, method, filed, etc.","A symbol pattern, expressed as a ""method"" pattern.","Path of the source file where a match found","FQName of the Class containing the symbol we search."
"match-annotation-001","JAVA","ANNOTATION","@org.springframework.boot.autoconfigure.SpringBootApplication","src/main/java/com/todo/app/AppApplication.java","com.todo.app.AppApplication"
Question: As we will have to pass a MatchID to ease the reconciliation between what we search and what we found within the CSV rows do you agree to create a FindRecipe class extending Recipe and that every find class will extend to contain fields like this ?
public class FindRecipe extends Recipe {
@Option(displayName = "Match id",
description = "ID of the matching tool needed to reconcile the records where a match took place",
required = false)
String matchId;
...
}
Remark: We could generalize too the fields used to search about: annotationPattern, methodPattern to pattern
Here is a PR (still draft) where I'm experimenting the idea reported here: https://github.com/snowdrop/migration-tool/pull/12 Test case is failing due to a diff as the visitor is passing twice within the method matching the annotation. Why ? I don't know
@timtebeek
I wouldn't create a shared base class for the recipe, nor a shared table for the search hits, as both would limit evolving the entries later.
nor a shared table for the search hits,
The more DataTables classes we will have, the more difficult it will be to reconcile the rows matching queries.
The MatchingDataTable could be viewed as similar or equivalent to the existing SourcesFileResults class able to generate for all the recipes CSV files. As this class and row will contain high level information for matching/Reconciliation, I don't think that it will break or limit evolutions.
Here is use case where a recipe has been created and includes several annotation search requests using the same recipe class
type: "specs.openrewrite.org/v1beta/recipe"
name: "dev.snowdrop.openrewrite.MatchConditions"
displayName: "Try to match a resource"
description: "Try to match a resource."
recipeList:
- dev.snowdrop.openrewrite.java.search.FindAnnotations:
pattern: "org.springframework.boot.autoconfigure.SpringBootApplication"
matchId: "rule-001-001"
matchOnMetaAnnotations: "false"
- dev.snowdrop.openrewrite.java.search.FindAnnotations:
pattern: "org.springframework.beans.factory.annotation.Autowired"
matchId: "rule-001-002"
matchOnMetaAnnotations: "false"
- dev.snowdrop.openrewrite.java.search.FindAnnotations:
pattern: "org.springframework.stereotype.Controller"
matchId: "rule-001-003"
matchOnMetaAnnotations: "false"
- dev.snowdrop.openrewrite.java.search.FindAnnotations:
pattern: "org.springframework.web.bind.annotation.GetMapping"
matchId: "rule-001-004"
matchOnMetaAnnotations: "false"
- dev.snowdrop.openrewrite.java.search.FindAnnotations:
pattern: "org.springframework.web.bind.annotation.DeleteMapping"
matchId: "rule-001-005"
matchOnMetaAnnotations: "false"
- dev.snowdrop.openrewrite.java.search.FindAnnotations:
pattern: "org.springframework.web.bind.annotation.PathVariable"
matchId: "rule-001-006"
matchOnMetaAnnotations: "false"
- dev.snowdrop.openrewrite.java.search.FindAnnotations:
pattern: "org.springframework.web.bind.annotation.PostMapping"
matchId: "rule-001-007"
matchOnMetaAnnotations: "false"
- dev.snowdrop.openrewrite.java.search.FindAnnotations:
pattern: "org.springframework.web.bind.annotation.RequestBody"
matchId: "rule-001-007"
matchOnMetaAnnotations: "false"
- dev.snowdrop.openrewrite.java.search.FindAnnotations:
pattern: "org.springframework.web.bind.annotation.ResponseBody"
matchId: "rule-001-008"
matchOnMetaAnnotations: "false"
When we run the following maven command
mvn org.openrewrite.maven:rewrite-maven-plugin:6.22.1:dryRun \
-Drewrite.activeRecipes=dev.snowdrop.openrewrite.MatchConditions \
-Drewrite.recipeArtifactCoordinates=org.openrewrite:rewrite-java:8.62.4,org.openrewrite.recipe:rewrite-java-dependencies:1.43.0,dev.snowdrop:openrewrite-recipes:1.0.0-SNAPSHOT \
-Drewrite.exportDatatables=true \
-Drewrite.configLocation=rewrite.yml
then we got as result a CSV file:
rewrite/datatables/2025-10-28_14-03-32-772/dev.snowdrop.openrewrite.java.search.MatchingReport.csv
including for each match a csv record
"Match ID","File's type","Symbol searched","A symbol pattern","Source file path","FQName of the Class"
"ID of the matching tool used to reconcile the information.","Type of the file where we look for.","Symbol about what we search about: annotation, import, method, filed, etc.","A symbol pattern, expressed as a ""method"" pattern.","Path of the source file where a match found","FQName of the Class containing the symbol we search."
"rule-001-008","JAVA","ANNOTATION","org.springframework.web.bind.annotation.RequestBody","src/main/java/com/todo/app/controller/TaskController.java","TaskController"
"rule-001-005","JAVA","ANNOTATION","org.springframework.web.bind.annotation.DeleteMapping","src/main/java/com/todo/app/controller/TaskController.java","TaskController"
"rule-001-006","JAVA","ANNOTATION","org.springframework.web.bind.annotation.PathVariable","src/main/java/com/todo/app/controller/TaskController.java","TaskController"
"rule-001-006","JAVA","ANNOTATION","org.springframework.web.bind.annotation.PathVariable","src/main/java/com/todo/app/controller/TaskController.java","TaskController"
"rule-001-003","JAVA","ANNOTATION","org.springframework.stereotype.Controller","src/main/java/com/todo/app/controller/TaskController.java","TaskController"
"rule-001-004","JAVA","ANNOTATION","org.springframework.web.bind.annotation.GetMapping","src/main/java/com/todo/app/controller/TaskController.java","TaskController"
"rule-001-004","JAVA","ANNOTATION","org.springframework.web.bind.annotation.GetMapping","src/main/java/com/todo/app/controller/TaskController.java","TaskController"
"rule-001-004","JAVA","ANNOTATION","org.springframework.web.bind.annotation.GetMapping","src/main/java/com/todo/app/controller/TaskController.java","TaskController"
"rule-001-004","JAVA","ANNOTATION","org.springframework.web.bind.annotation.GetMapping","src/main/java/com/todo/app/controller/TaskController.java","TaskController"
"rule-001-007","JAVA","ANNOTATION","org.springframework.web.bind.annotation.PostMapping","src/main/java/com/todo/app/controller/TaskController.java","TaskController"
"rule-001-009","JAVA","ANNOTATION","org.springframework.web.bind.annotation.ResponseBody","src/main/java/com/todo/app/controller/TaskController.java","TaskController"
"rule-001-001","JAVA","ANNOTATION","org.springframework.boot.autoconfigure.SpringBootApplication","src/main/java/com/todo/app/AppApplication.java","AppApplication"
"rule-001-002","JAVA","ANNOTATION","org.springframework.beans.factory.annotation.Autowired","src/main/java/com/todo/app/controller/TaskController.java","TaskController"
"rule-001-002","JAVA","ANNOTATION","org.springframework.beans.factory.annotation.Autowired","src/main/java/com/todo/app/service/TaskServiceImpl.java","TaskServiceImpl"
Observation
- Without a matchid, it will be needed that the process reading the records will search again if the line contains one of the annotations (or any other symbol: field, method, etc) searched to have a match
found. Using amatchIdassociated with the initial query (= what finally execute the mvn cmd) simplifies the process to read the records and reports or not that we have a match found if the matchId's query is the same as the one of the record - As an annotation can appear several times within a class (example: GetMapping, PathVariable), then the same matchID can appear several times. Until now we can assume that if we have at least one record to report
match=found. We can improve later how the information is reported here to include as this is the case with a language server the location (position, etc) of thesymboldiscovered within a java file, etc - A convention should be also adopted and defined to allow to easily pickup the correct CSV file from the generated folders. Example if the recipe class name is
FindAnnotationsthen we could use as DataTable class nameAnnotationsReportto find the CSV file =>rewrite/datatables/DATE_TIME/<ROOT_PACKAGE_NAME>.<TYPE_SEARCHED>.search.AnnotationsReport.csv
If the recipe datatable supports a matchId, then we could when we match/reconcile the information shows such a table result
=== Code Analysis Results (Improved Formatting) ===
+----------------------------------------------+-----------------------+-------+-----------------------------------------------------------------------------------------+
| Rule ID | Source to Target | Found | Information Details |
+----------------------------------------------+-----------------------+-------+-----------------------------------------------------------------------------------------+
| springboot-annotation-notfound-00000 | springboot -> quarkus | No | No match found |
+----------------------------------------------+-----------------------+-------+-----------------------------------------------------------------------------------------+
| springboot-to-quarkusmain-annotation-00000 | springboot -> quarkus | Yes | File: 2025-10-28_19-14-27-976/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 5 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.boot.autoconfigure.SpringBootApplication |
| | | | Match ID: 87bbf766-eda6-4587-b258-2e932fa6b007 |
+----------------------------------------------+-----------------------+-------+-----------------------------------------------------------------------------------------+
| springboot-replace-bom-quarkus-0000 | springboot -> quarkus | Yes | File: 2025-10-28_19-14-18-299/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 5 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.boot.autoconfigure.SpringBootApplication |
| | | | Match ID: dca42c82-ca2d-4d95-96f9-149cdc1c42a8 |
+----------------------------------------------+-----------------------+-------+-----------------------------------------------------------------------------------------+
| springboot-add-class-quarkus-00000 | springboot -> quarkus | Yes | File: 2025-10-28_19-14-23-115/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 5 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.boot.autoconfigure.SpringBootApplication |
| | | | Match ID: ff71cb64-18ae-46e9-bc74-acbd9e8d84ed |
+----------------------------------------------+-----------------------+-------+-----------------------------------------------------------------------------------------+
| springboot-to-quarkus-rest-annotations-00000 | springboot -> quarkus | Yes | File: 2025-10-28_19-14-32-768/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 5 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.stereotype.Controller |
| | | | Match ID: fe95e921-5a72-49eb-b79b-0dda2794df35 |
| | | | --- rewrite --- |
| | | | File: 2025-10-28_19-14-37-472/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 5 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.beans.factory.annotation.Autowired |
| | | | Match ID: 7f9dc4c2-be40-449d-ab41-54c6e52f6ae7 |
| | | | --- rewrite --- |
| | | | File: 2025-10-28_19-14-37-472/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 6 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.beans.factory.annotation.Autowired |
| | | | Match ID: 7f9dc4c2-be40-449d-ab41-54c6e52f6ae7 |
| | | | --- rewrite --- |
| | | | File: 2025-10-28_19-14-42-480/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 5 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.web.bind.annotation.GetMapping |
| | | | Match ID: a4619194-29a1-4d68-a792-8a0d83e50bd5 |
| | | | --- rewrite --- |
| | | | File: 2025-10-28_19-14-42-480/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 6 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.web.bind.annotation.GetMapping |
| | | | Match ID: a4619194-29a1-4d68-a792-8a0d83e50bd5 |
| | | | --- rewrite --- |
| | | | File: 2025-10-28_19-14-42-480/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 7 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.web.bind.annotation.GetMapping |
| | | | Match ID: a4619194-29a1-4d68-a792-8a0d83e50bd5 |
| | | | --- rewrite --- |
| | | | File: 2025-10-28_19-14-42-480/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 8 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.web.bind.annotation.GetMapping |
| | | | Match ID: a4619194-29a1-4d68-a792-8a0d83e50bd5 |
| | | | --- rewrite --- |
| | | | File: 2025-10-28_19-14-47-469/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 5 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.web.bind.annotation.DeleteMapping |
| | | | Match ID: aa2e7f2e-6d4e-4eaa-9e7f-805d124c3f28 |
| | | | --- rewrite --- |
| | | | File: 2025-10-28_19-14-52-247/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 5 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.web.bind.annotation.PathVariable |
| | | | Match ID: 86ee0d6f-b57b-4beb-936d-14a01e26b2e5 |
| | | | --- rewrite --- |
| | | | File: 2025-10-28_19-14-52-247/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 6 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.web.bind.annotation.PathVariable |
| | | | Match ID: 86ee0d6f-b57b-4beb-936d-14a01e26b2e5 |
| | | | --- rewrite --- |
| | | | File: 2025-10-28_19-14-57-575/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 5 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.web.bind.annotation.PostMapping |
| | | | Match ID: 6a81a425-3bc8-4fb6-8d87-82393a4e6677 |
| | | | --- rewrite --- |
| | | | File: 2025-10-28_19-15-02-693/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 5 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.web.bind.annotation.RequestBody |
| | | | Match ID: cf256bbc-e682-4d3c-8a18-329517feb2da |
| | | | --- rewrite --- |
| | | | File: 2025-10-28_19-15-07-767/dev.snowdrop.openrewrite.java.table.AnnotationsReport.csv |
| | | | Line: 5 |
| | | | Pattern: JAVA.ANNOTATION |
| | | | Type: org.springframework.web.bind.annotation.ResponseBody |
| | | | Match ID: 47497b33-c09a-4bb4-9dd3-2fee053eef4e |
+----------------------------------------------+-----------------------+-------+-----------------------------------------------------------------------------------------+
```
Thanks for the detailed explanation; I think in this case your needs are starting to diverge quite a bit from what we originally built our search recipes for: in our case they are meant to add informational search markers which we show in our UI (or failing that print into source files), whereas you're looking to do a wide search and build a report on what was found. Some of the features you've described above like match ID and line number aren't supported at the moment, and would be quite a substantial set of changes for one specific use case.
On the other hand I do feel as if what you're looking for can be achieved with a handful of custom search recipes that you maintain yourself, as opposed to us adopting those changes into all our search recipes. You could fork the search recipes that we have to add your common data table as desired, and then build your reports based on that.
Separately from our side we'll continue to aim for code changes in relation to the switch from Spring to Quarkus, as already seen in https://github.com/openrewrite/rewrite-spring-to-quarkus/. Both can approaches can be developed in parallel and complementary. I know the liberty team at IBM has similarly built tooling with a focus on building reports, which can then optionally suggest recipes to make changes based on the findings.
Could that be a way forward for you without changes to the core here?
On the other hand I do feel as if what you're looking for can be achieved with a handful of custom search recipes that you maintain yourself,
This is what I started to do part of our POC ;-)
as opposed to us adopting those changes into all our search recipes. You could fork the search recipes that we have to add your common data table as desired, and then build your reports based on that.
As you know forking a project is not a solution as that costs a lot to maintain such a code separately and if there is no roadmap defined to include at some point new features, than projects will continue to diverge or be abandoned finally !
Remark: It is nevertheless important that such a discussion take place and at least that this feature request becomes (I hope) part of a future iteration of openrewrite even if this is in 6 months, 1 year , etc as the added value of the proposition exists, is relevant and will interest more than one user and of course could benefit to openrewrite too
I certainly did not mean to fork the whole project! :) Just to create your own copy of the handful of search recipes you're interested in to add the data table records, much like you're already doing in https://github.com/snowdrop/springboot-to-quarkus-migration-guide
On the other hand I do feel as if what you're looking for can be achieved with a handful of custom search recipes that you maintain yourself, as opposed to us adopting those changes into all our search recipes. You could fork the search recipes that we have to add your common data table as desired, and then build your reports based on that.
Forking the project is not a solution like also to copy on my project some of the Findxxxx classes.
We can for sure extend the openrewrite classes to add the missing field matchId like the transient DataTable but the part of the code where we would like to insert the Row record within the one of the visit method will be challenging.
We could use AOP or CDI interceptors (if you enjoy quarkus AOP - https://quarkus.io/guides/cdi#interceptors) but again that will require many lines of code to maintain, to support to visit each type of module: java, maven, xml, properties, etc and to have add also an annotation part of the openrewrite find class potentially too
wdyt ? @timtebeek
My time is limited this week, so my answer will be short but not intended to be coarse or unkind: I don't see us change every search recipe we have to add a matching parameter + data table. I don't foresee an easy way to support the same by extending recipe classes in a maintainable way.
Our focus has always been on producing code changes through recipes, as well as doing quick but thorough analysis through individual recipe runs; not by running a composite of recipe instances together to produce a larger report.
The way forward I see for your use case in the short term is to write your own search recipes and data tables to support your use case, as it's rather unique. Would love to collaborate of course where our interests align, such as the recipes in rewrite-spring-to-quarkus, but in the short term I don't see us make sweeping changes to our core recipes to support the reporting use case described here.