Seamless interoperability with jupytext

[st--]

trafficstars

I've been writing some examples as jupyter notebooks, saved using jupytext as .jl files. I find it helpful to write equations in the jupyter notebook to get instant rendering - but unfortunately I can't directly get this and use Literate.jl because of the discrepancy between using $$ $$ or ```math ``` for display maths, and the extra -- coding: utf-8 -- at the top of the .jl file that doesn't get stripped away. Would you be willing to consider including something like a `jupytext::Bool`` flag that would handle these minor discrepancies and allow seamless use of the same file both for editing in jupyter and for turning into documentation? If you're up for this in general, I'd be happy to discuss how this could look & contribute a PR.

[fredrikekre]

Can you post an example of the jupytext output?

[st--]

https://github.com/JuliaGaussianProcesses/KernelFunctions.jl/blob/2d84c74da733ab74665c141b0698da01a08cd1f2/examples/gaussianprocesspriors.jl and https://github.com/JuliaGaussianProcesses/KernelFunctions.jl/blob/2d84c74da733ab74665c141b0698da01a08cd1f2/examples/kernel_ridge_regression.jl though I already made some changes to get it to play along a bit better. Notable differences are that Literate.jl exclusively uses # at the line start as "not a code example" and you have to use ## to actually get a comment, whereas jupytext does splitting into cells (and assigning as markdown/code) first. Jupytext also has #- and #+ to denote larger code cells with blank lines inbetween (though there is a slight format variant that does away with this).

[st--]

The rendered (Literate.jl/Documenter.jl) output is here: https://juliagaussianprocesses.github.io/KernelFunctions.jl/previews/PR234/examples/gaussianprocesspriors/ and https://juliagaussianprocesses.github.io/KernelFunctions.jl/previews/PR234/examples/kernel_ridge_regression/ (I'm not sure why the first one doesn't actually render the figures, whereas it works fine for the second one :thinking:)

[fredrikekre]

Thanks. It is strange that some math work and other don't.

[fredrikekre]

To get rid of the # -*- coding: utf-8 -*- header you can use a preprocess function, see https://fredrikekre.github.io/Literate.jl/v2/customprocessing/. Something like

function remove_header(str)
    str = replace(str, r"^(.*coding: utf-8.*)$"m => ""; count=1)
end

should work I think.

[st--]

Handling ```math ... ``` vs $$ ... $$ still needs extra parsing though, likewise for cell splitting / #- ... #+?

[terasakisatoshi]

To get rid of the # -*- coding: utf-8 -*- header you can use a preprocess function, see https://fredrikekre.github.io/Literate.jl/v2/customprocessing/. Something like

function remove_header(str)
    str = replace(str, r"^(.*coding: utf-8.*)$"m => ""; count=1)
end

should work I think.

Thank you. This workaround could get rid off ut8-.... 👍

[stevengj]

Note that it is also easy to parse Jupyter notebooks and output .jl files yourself, because they are just JSON.

See this discourse post for example — replace text/plain with text/markdown in the output line to output Markdown code for the comments rather than converting to plain text.

Would it be possible to support this directly in Literate.jl, e.g. allowing one to use Jupyter notebooks directly as the input format?