redbaron icon indicating copy to clipboard operation
redbaron copied to clipboard

Middle-ground betwixt `ast` and `baron` for parsing source code, and emitting only what changed in FST from AST

Open SamuelMarks opened this issue 4 years ago • 1 comments

What I'm imagining is using ast to find and modify nodes, and when ready to emit (unparse) align with the FST so that only what the AST has changed will be modified.

Some caveats:

  • Cannot delete nodes (I assume)
  • Maybe difficulty in converting single line to multiline, and multiline to single line

What do you think?

I am modifying AST nodes, changing: docstrings; ast.Assign; ast.AnnAssign; and ast.FunctionDef/ast.AsyncFunctionDef. Inferring types, adding them as a type comment xor annotation; converting between docstring formats (incl. adding/remove types); and updating the return attribute of a function definition.

Currently my library only uses ast and inspect for this… which means it can support 3.6, 3.7, 3.8, 3.9, 3.10 & 3.11. https://github.com/SamuelMarks/cdd-python

…but adding/removing type annotations, for example, has the [unexpected] side-effect of removing all comments and reducing whitespace.

Tossing up betwixt rewriting my library around redbaron | LibCST and hacking together a basic parser & emitter to replace ast.parse and ast.unparse with some line-aware copypasta.

Related: #207

Thanks for sharing your perspective

SamuelMarks avatar Dec 16 '21 19:12 SamuelMarks

FYI: this project is not actively updated. Parso is another FST that I've been pointed towards in the past (although I haven't checked it in detail).

Erotemic avatar Dec 16 '21 22:12 Erotemic