feat: plaintext support

[lukastk]

Closes https://github.com/AnswerDotAI/nbdev/issues/1498

Adds support for exporting the following plaintext formats as if they were regular .ipynb files:

• percent
• light
• sphinx
• myst
• pandoc

See this for details on the above plain-text notebook formats.

Running nbdev_export now converts, by default, files of extension *.{pct.py,lgt.py,spx.py,myst.md,pandoc.md,ipynb}. See this for a demonstration (and some documentation) on the new exporter.

The feature requires jupytext and nbformat as (optional) dependencies. They will only be required if users attempt to export non-ipynb files.

[lukastk]

Note: In this commit I change the default behaviour of nbdev_export to also include the plaintext formats by default. This can be rolled back in case it's deemed too drastic a change. In which case, plaintext formats can be exported by running, for example:

nbdev_export --file_glob '*.pct.py'

[hamelsmu]

Thanks for providing the blog post with the motivation for this PR.

Plain text formats don't capture outputs which are required to render nbdev docs so I'm afraid this is not a good fit. Please let me know if I'm wrong or I am overlooking something!

[lukastk]

Hi @hamelsmu. Thanks for taking the time to look through the PR and issue.

I agree that plaintext formats do not capture outputs, but I don't think these are required as such to render them via Quarto, which itself has support for rendering notebook-formatted .py files (see this).

I take your point that we often want to render the outputs in the docs as well, although for some use-cases (like mine) it's less necessary. One option would be to add a pre-processing step where the plaintext files are converted to .ipynb notebooks, and then executed. This way the outputs of the code cells would be rendered. I'd be happy to take a look into this if you think it's a good idea.

[jlopezpena]

I have used jupytext before for docs, and it is possible to get it to run the notebooks before generating the documentation, jupyterbook does so in a very nice way. I personally like my documentation rendered (and published!) as part of the CI/CD process, and since the notebooks need to run anyway for the test part, there is no additional overhead.

In my opinion, there is a great benefit on not having to deal with the outputs under version control, particularly when they are binary objects (like images). Besides the pain to have to use a paid tool for proper code reviews, changes in binary files increase the size of the git history massively, making it quickly become annoying to deal with

[jlopezpena]

PS Quarto also has the option to execute cells when rendering: https://quarto.org/docs/computations/execution-options.html

[lukastk]

Thanks @jph00 for your comments.

Let's make this opt in, disabled by default. You can add a setting for it. I've added some minor formatting requests.

Sounds good, will do!

It's unusual to have a separate "tests" notebook. Can this be integrated into the docs instead, creating tests that also illustrate the behavior?

Sure, can integrate these into the docs.

[jph00]

Looks like this is abandoned so closing for now.