Use markdown extension API

[ZedThree]

There are three functions in utils.py that add Ford-specific features to the markdown:

• sub_macros: replace |aliases| with literal text
• sub_notes: replace @notes with div element boxes
• sub_links: replace [[links]] with links to other pages

These could all be replaced with the Markdown extension API. It's not a massive amount of work, but it would involve some restructuring of the code, as well as possibly some backwards incompatible changes to how the markdown is processed (see https://github.com/Fortran-FOSS-Programmers/ford/issues/223#issuecomment-918193594)

[mbraakhekke]

I'd be willing to work on this and submit a PR. I'd like to contribute to FORD and I already dug into the code for sub_links() for #370. Hopefully it's also possible to handle this issue with the Markdown extension API.

[ZedThree]

Thanks @mbraakhekke, that would be lovely! See #223 for one possible stumbling block. I think if it's not possible to support @notes/todo etc in certain places, then that's probably a price worth paying. We'll just bump the major version.

I can push my initial attempt to a branch, if you'd like to use that as starting point/inspiration etc?

[mbraakhekke]

I can push my initial attempt to a branch, if you'd like to use that as starting point/inspiration etc?

Yes please. Thanks!

[ZedThree]

Pushed to https://github.com/Fortran-FOSS-Programmers/ford/tree/markdown-extensions

There's a new file admonition.py which I took from the Python markdown project and modified. I was testing it by hand, so there's no changes to actually use that.

GitHub

GitHub - Fortran-FOSS-Programmers/ford at markdown-extensions

Automatically generates FORtran Documentation from comments within the code. - GitHub - Fortran-FOSS-Programmers/ford at markdown-extensions

[mbraakhekke]

So I finally found some time to work on this. On studying the code, I realized that these three functions are used on both the markdown pages and the pages generated from the Fortran source files. Since the latter are not in markdown, the markdown extension API will not work for this. This means that the original functions will have to stay.

Given this, I wonder if it is worth the effort of creating markdown extensions to do this specifically for the markdown pages.

[ZedThree]

I'm not sure what you mean -- all the docstrings from the Fortran source are passed through markdown in FortranBase.markdown: https://github.com/Fortran-FOSS-Programmers/ford/blob/d74fd9a4a6a68985d52713413ba2bde4c44ac8ca/ford/sourceform.py#L327-L328

which is called (via FortranProject.markdown) using the Markdown object initially created here: https://github.com/Fortran-FOSS-Programmers/ford/blob/d74fd9a4a6a68985d52713413ba2bde4c44ac8ca/ford/init.py#L318-L324

[mbraakhekke]

Ah, apologies. turns out I didn't study the code thoroughly enough. I'll start working on it then.

[ZedThree]

No worries, it's always tricky to follow the logic on an unfamiliar codebase!

[mbraakhekke]

I'm working on the MD extension for replacing sub_notes() based on the modified admonition.py you provided.

This FORD syntax for notes:

@note
Some text.
@endnote

is quite different from that for admonitions in markdown:

!!! type "optional explicit title within double quotes"
    Any number of other indented markdown elements.

    This is the second paragraph.

In #223 you suggested to implement a kind of hybrid syntax. I wonder what's the right path now. Modifying admonition.py to support the original FORD syntax might be quite a bit of work, so if we decide to implement your idea I'd make sense to do this now. However, that should be discussed first since it's a breaking change, requiring users to modify their documentation. Also it may be possible to solve #223 while maintaining the original FORD syntax (I haven't really thought about this yet). Alternatively, we could implement a configuration option to switch between the old or new syntax.

[ZedThree]

I agree that we should explore non-breaking changes first. Hiding the breaking change behind a configuration option is a good idea if that isn't possible. We could then print a deprecation warning if the old syntax is encountered, and flip the switch in a new major version.

One idea I had was to use a (markdown) preprocessor to indent the text between the @note and @endnote, and remove the @endnote. This initial pass would then get the text into the correct state for the main pass to work. I think the difficulty was because the @endnote is optional, I couldn't work out when the note block was supposed to end. The current implementation looks for either a closing paragraph tag </p> or @endnote.

My suspicion is that it's just not possible to support the current syntax with an MD extension, but conversely, there are other situations that we would be able to support. I think the balance is in favour of the admonition syntax though.

[mbraakhekke]

Since we're really talking about #223 now, I'll reply in that issue.