ruby
Modifying AST and generating source code
Forgive my ignorance but I'm trying to use Prism to modify Ruby source (add nodes) and then regenerate the source to be interpreted. I'm using Prism.parse(text) and then implementing a Prism::Visitor to potentially modify the nodes as they are visited. I assume this is possible but I haven't really seen any examples of it. Finally I would like to 'unparse' the AST back into source code.
The Python equivalent is indeed unparse which is where I'm taking many of the ideas. They also have a NodeTransformer which is allowed to modify the nodes in the visit pattern. I don't see equivalents in Prism but perhaps you're always allowed to modify the nodes ... this is Ruby afterall.
Thanks for any help pointing me in the right direction!
If it can't be done to generate the source, I don't require the generated source code as long as I can run the modified AST result. Something like eval the modified AST. Not sure if that is possible.
Not quite sure why you pinged me on this 😅 I guess because I worked on the parser translator here, and RuboCop would need something like this as well in the future. But right now I haven't invested too much time into thinking about how RuboCop would make use of prism without a middleman. No ideas from me, sorry
@Earlopain thanks ... I was just looking for advice for someone knowledgeable about Prism to weigh in on whether this was even possible right now.
@jmthomas Evenutally unparser will support prism see:
- https://github.com/ruby/prism/issues/3553
- https://github.com/mbj/unparser/pull/387
- https://github.com/mbj/unparser/issues/385
This is done via the translation layer to the parser AST, but I plan to drop to native as soon I do not have to support 3.2 anymore.
Closing this since it is more a question, but the answer is yes, you can use a visitor to modify the AST, or even build a new AST. And they use something unparser to generate the ruby code again.
unparser depends on the parser gem which I think makes it rather undesirable in combination with prism. I haven't checked it out yet but somewhat doubt that using the native prism ast would even work with unparser. This is not such a big deal for folks that currently use one of the translation layers since they can just continue to use what they already use (be that unparser, Parser::TreeRewrite or something else). But when you use prism bare it is a bit of a bother.
Previously kddnewton expressed desire to add such an api to prism itself here: https://github.com/whitequark/parser/issues/1046#issuecomment-2491891823
Reading that thread, https://github.com/whitequark/parser/issues/1046#issuecomment-2492526529 seems also relevant.
I think unparser should/could work, by parsing, mutating the Prism AST, applying the translation to parser AST, use unparser. Obviously not the most efficient though.
I think unparser is probably what I'm looking for. Looks like they are waiting for a new release of Prism to incorporate a few fixes before they support Ruby 3.4 and beyond.