API to preserve format of nodes unchanged by codemods

[jedwards1211]

I'm on the hunt for better ways to prevent irrelevant formatting changes in codemods... AFAIK the most popular tool for this right now is recast (used by JSCodeshift), but it's slow to add support for new syntax features, and I've seen buggy output from it various times over the years.

Reformatting the transformed code with prettier helps, but it's not enough because unchanged objects that were previously on one line can get broken up into multiple lines, and so forth. One of the most basic solutions would be deterministic output, but that wouldn't be good at preserving blank lines or one-line groups.

If only prettier exported its printers, preserving format after a codemod would be fairly simple, theoretically:

• Make a custom printer that wraps another printer
  • When printing a given node:
    • if it (and all its descendants) were unchanged by the codemod:
      • output the original source for the node verbatim
    • otherwise, let the wrapped printer print the node
      • if any of its descendants were unchanged by the codemod, this printer will use the original source for them too
• Do a second pass to reformat the code with the usual printer

This would be one potential benefit of exporting internal plugins printers.

I was able to do something similar to this with @babel/generator via a hack that allowed me to wrap its print node function, and so far the results have been very good. But @babel/generator doesn't export its printer API either so I had to use a hack to get at it.

The resulting indentation can be a little off, but the second pass through prettier fixes that.

Maybe another theoretical option would be to parse the source into prettier's internal AST format, manipulate that AST, and then call formatAST on it, but formatAST unfortunately isn't public API, and the tools I'm concerned with might require a complete redesign to operate on prettier AST nodes, I don't know.

[pieterv]

What you are describing is something we have done for a while with prettier, check out the hermes-transform package linked below (this is designed for Flow use cases so it's unlikely to work directly for you but the approach should).

https://github.com/facebook/hermes/blob/main/tools/hermes-parser/js/hermes-transform/src/transform/print.js

Essentially the way it works is you parse the AST, manipulate it, then pass the AST to prettier to print via the plugins API. There are some tricks you need to use to get the AST to work with prettier directly, like attaching comments before AST manipulation and ensuring empty location information is present for new nodes. In general it works well as long as your codebase is consistently formatted by prettier. It's probably possible to setup a similar system using Babel's parse and transform API's.

[jedwards1211]

Interesting... I assume the AST had to be parsed via a prettier plugin somewhere outside that file you shared though?

I was looking for an option for printing an AST that came from Babel parsing without being decorated by prettier.

I'm not using babel's transformer either, I've written my own pattern-based search and replace tool called astx and I'm looking for more reliable ways to preserve format with that (or with jscodeshift)

[pieterv]

No the AST just needs to be compatible with what prettier expects, most parsers use the estree format. We use the Hermes parser which is not internal to prettier. The babel AST format is non standard but it is used internally in prettier as one of the parsing options so i assume it could work.

[jedwards1211]

I'm confused how your example approach would preserve things like object literals being on one line vs. multiple lines then. Unless prettier extracts from the original source for any node that has a location, and you've deleted the location from modified nodes? Or do you just not care about preserving onelineness?

[pieterv]

Yeah prettier prints based off the source position which is embedded in the AST. Our tool has a way to modify nodes without removing the source location. It’s just new nodes that wouldn’t have location information and I have found the prettier defaults to work well.

[jedwards1211]

I see...but what does prettier do if the parent node has a source position, but its child nodes have been modified? Or even just moved around, so none of them lack a source location, but the output will have to differ from the original source?

[jedwards1211]

Okay I started to get something working last night, big thanks for your tips and giving me a link to how you do it!

[jedwards1211]

Hmmm, so far it seems to do a good job preserving comment formatting, but it's tending to reformat unchanged single-line objects into multiple lines. Does that likely mean I'm doing something wrong?

[pieterv]

Yeah it means the location information is not being set on the AST for prettier correctly. Try inspecting your AST nodes, from memory prettier looks at the a range array. This is the format we use https://github.com/facebook/hermes/blob/main/tools/hermes-parser/js/hermes-parser/src/HermesToESTreeAdapter.js#L33-L39. You can inspect prettier's printing logic to see what it expects when printing objects.