Documenter.jl icon indicating copy to clipboard operation
Documenter.jl copied to clipboard

WIP: Use MarkdownAST package

Open mortenpi opened this issue 1 year ago • 0 comments

This replaces the Markdown2 module with MarkdownAST as the representation of the Markdown documents as AST (in doctests). It adds MarkdownAST and AbstractTrees as a dependency here (the latter for its tree-walking algorithms). For merging we need to actually tag a version of MarkdownAST, which is currently unregistered.

For a little bit of history: I intended the Markdown2 module to be a clean and documented version of the Markdown standard library AST that we could use throughout Documenter. Using it for doctests was just a proof-of-concept, but it never went beyond that. So this is take two.

However, I think the approach CommonMark takes (single Node type; leading to consistency and type stability) is far superior to the approach by Markdown and Markdown2, and so MarkdownAST lifts the bare minimum out of CommonMark into a separate lightweight package that can act as an interface. In the future I hope users and extension packages can feed their own AST into Documenter directly, making it easier to have more dynamic pages (i.e. no need to generate intermediate Markdown files etc.).

Converting whole of Documenter to MarkdownAST is a pretty big refactoring. The current plan is to do it in roughly the following PRs:

  1. This PR adds the dependency and uses it in the DocTests module (basically a simple drop in replacement for Markdown2).
  2. Develop a MarkdownAST Node-based representation of the whole Documenter document, implement a conversion function from the current Document object to that representation, and then convert one of the writers to use that new representation (LaTeXWriter might be easier as a proof of concept).
  3. Convert the other writer (HTMLWriter) too.
  4. Convert the earlier build steps, so that they would mutate a MarkdownAST tree, rather than working on the Markdown-based representation.

This currently uses the mp/abstractrees branch of MarkdownAST.

mortenpi avatar Jul 31 '22 05:07 mortenpi