p5-Devel-IPerl icon indicating copy to clipboard operation
p5-Devel-IPerl copied to clipboard

Convert notebooks to POD

Open zmughal opened this issue 10 years ago • 11 comments

  • Convert images (to files?).
  • Add a link to nbviewer version.
  • Integrate with GitHub.
  • options to include stderr/stdout

zmughal avatar Feb 08 '15 18:02 zmughal

This is an awesome idea!

demianriccardi avatar Feb 08 '15 18:02 demianriccardi

An additional thing that can be done is to have POD input cells in the notebook itself. I'm looking into that, but it might require having a JavaScript POD parser. @ingydotnet wants to write one using Pegex.

Yet another approach would be to have the REPL input scan for POD in the code input cell and output HTML along with with the code output.

zmughal avatar Feb 09 '15 23:02 zmughal

Do you think it would be tricky to implement a Perl nbviewer within metacpan? I'd love to have them linked into the docs.

demianriccardi avatar Feb 09 '15 23:02 demianriccardi

Pegex works in JS thus a Pegex Pod grammar would work in JS as well as Perl. (Any language with a regex engine).

ingydotnet avatar Feb 10 '15 03:02 ingydotnet

More info on how to do this from the IPython-dev mailing list:

You can have raw cells with a specified mime type. By default, these will display in the notebook as plain, unhighlighted text, but you could write an extension that watched for raw cells with a specific mime type and then rendered them. A JS POD parser would be required for that, though.

Exporting them with nbconvert should be easier, though - you can write Jinja templates and filters that will, for instance, run those output cells through a command line program to render them.

zmughal avatar Feb 10 '15 19:02 zmughal

@demianriccardi, I don't know if MetaCPAN could support having nbconvert directly working in the MetaCPAN renderer, but if notebooks can be converted to POD, then we wouldn't have to worry about that. Actually, as an approximation, I could probably experiment with first using nbconvert and sticking the HTML in the POD. There will be some elements that won't render (see the use of HTML::Restrict here).

zmughal avatar Feb 11 '15 05:02 zmughal

I suspect that MetaCPAN can't support nbconvert directly if it involves executing 3rd party code from the notebook. Converting notebooks to POD at dist build time seems like the right way about this and ensures that folks without nbconvert or IPython Notebook can view the notebooks when they download the tarball.

Since nbconvert can output Markdown, and Markdown::Pod can convert Markdown to POD, this should be a matter of putting them together. :) (It may be complicated by fidelity issues, of course, but presumably those are fixable.)

Finally, perhaps it makes sense for MetaCPAN to support rendering Markdown files as POD automatically via Markdown::Pod.

tsibley avatar Feb 11 '15 07:02 tsibley

@tsibley, thanks for the Markdown::Pod tip! Going the route of nbconvert → Markdown → POD didn't work, but I think I have the start of something. I'm not sure how robust it is to different HTML blocks nor do I know what to do in cases that won't work on MetaCPAN (like <iframe>s), but here we go:

http://rawgit.com/zmughal/iperl-to-markdown-to-pod-test/master/20150209_IPerl_display_demo_hackitup.html

Code: https://github.com/zmughal/iperl-to-markdown-to-pod-test/blob/master/hack-it-up.pl.

zmughal avatar Feb 11 '15 08:02 zmughal

Currently, MetaCPAN scrubs data: URIs and <font> tags, so both the image output and terminal output does not show up properly on MetaCPAN: example.

It looks like the data: URI should work. As for the <font> tags, in this case, I suppose a good solution would be to replace it with <code> tags. That would be more semantic.

Edit: the data: URIs should now be fixed by https://github.com/CPAN-API/metacpan-web/pull/1457.

Edit: Now the terminal output uses a <span style="..."> instead of a <font> tag. The style is needed so that the line-height leaves no gaps when drawing the background.

zmughal avatar Feb 15 '15 21:02 zmughal

As I continue to use the notebook to write up experiments, I realised that the LaTeX would not show up nicely on MetaCPAN because it uses MathJaX. @jberger has done some investigation into this area before: http://mailman.jach.hawaii.edu/pipermail/pdl-porters/2013-January/005281.html, http://pdlporters.github.com/?docs=Acme::Pod::MathJax, https://metacpan.org/module/Acme::Pod::MathJax.

I believe some folks have MathJaX enabled locally, so that might be an easy way out.

zmughal avatar Feb 17 '15 01:02 zmughal

:+1:

Your idea and execution of it is wonderful. Please keep up doing the awesome work :100: / :100:

azawawi avatar Feb 19 '15 09:02 azawawi