Extensible Renderers Connected to Kernels

[saulshanabrook]

Is your feature request related to a problem? Please describe.

I would like to be able to render interactive Altair visualizations backed by large databases using ibis-vega-transform in Jupyter Book.

Describe the solution you'd like I would like to gauge the interest in helping design and adding support in Jupyter Book for this proposal on extensible Jupyter renderer.

The TLDR is "Standardize a MIME type where the content is the URL pointing to a JS script that export a function that frontends can call to render the output, passing in the target DOM node, as well as the mime bundle, and optionally some functions they can use to connect to comms on the kernel"

The goal is that then different Jupyter frontends will not have to build bespoke support for different renderers, and different rendering libraries will not build separate rendering capabilities for all frontends.

That specification still needs some more review and TLC, but we have already gotten some feedback from folks who work on nteract and colab.

Describe alternatives you've considered

One alternative is to instead just build in support for this render to Jupyter Book directly. This doesn't seem great because this renderer isn't that popular and I don't want to tie our development processes together.

Another option, would be to let Jupyter Book be launched with support for different rendering extension, similar to how in JupyterLab you install extensions to register new renderers. This does allow you to possibly ship a smaller total bundle, but introduces a lot of complexity to the build system. Also, it means that notebooks will only render in Jupyter Book setups that have been built for them.

A third option is to just support Jupyter Widget, and force any third party renderers to go through that framework, which is what Voila does currently. I don't love this alternative, because ipywidgets comes with a lot of abstractions over comms, which I, for example, don't need in my renderer. It seems cleaner to specify a smaller minimal interface, that only relies on Jupyter's built in primitives, comms, output data, and updating outputs.

Additional context

I talked with this a bit with @choldgraf over Gitter just now. He encouraged me to open an issue and pointed me to this example of running jupyter book with a live kernel.

Supporting this proposal, would not require support for a live kernel, we could still support custom renderers and not pass in those args (it's mean to support graceful degradation in environments like this), but for our use case we would require a live kernel to be active.

[welcome[bot]]

Thanks for opening your first issue here! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.
If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).
Welcome to the EBP community! :tada:

[choldgraf]

I think this would really simplify our ability to facilitate non-trivial visualization in Jupyter Book. It would be great if we, as a tool with no knowledge of the environment that generated a MIME bundle, could know how to display the contents of that MIME bundle in a self-contained fashion 👍

otherwise, we're gonna have to create one-off exceptions and/or documentation items for how people can build jupyter books that know how to display all the various visualization libraries out there

also just to be clear - this MIME bundle standardization effort is something you and others are interested in exploring as well? It seems like a much more largely-scoped thing than Jupyter Book. Have you considered writing up a JEP for this?

[saulshanabrook]

otherwise, we're gonna have to create one-off exceptions and/or documentation items for how people can build jupyter books that know how to display all the various visualization libraries out there

Yep exactly.

also just to be clear - this MIME bundle standardization effort is something you and others are interested in exploring as well? It seems like a much more largely-scoped thing than Jupyter Book. Have you considered writing up a JEP for this?

Yeah, I have considered that, seems like a good idea! I don't have the bandwidth right now to push forward on that process.

[choldgraf]

I should say:

adding support in Jupyter Book

I think this is a great idea that we can try out. The place to do this is in MyST-NB. Here's roughly what happens:

• When a notebook is executed as part of a Sphinx build, we put the whole output mimebundle embedded in the document. (here's where we parse the notebook originally and here is where we dump in the MIME bundle to be transformed later)
• Later on, we parse the document for all of these MIMEBundles, and then replace them with the relevant docutils objects to be displayed (here is the class that does this, and here is where we define render priority for MIME types)
• These are then rendered into HTML at the end by Sphinx

So I think if we wanted to essentially support a special MIME bundle structure for certain kinds of outputs, the change would be somewhere around the output to nodes functions

[choldgraf]

I'm gonna transfer this one over to MyST-NB, as that is where the improvement would need to be. Curious what @chrisjsewell thinks about the proposal - I like the idea because (if the spec is adopted by other libraries) it could simplify our maintenance burden for these other libraries quite a bit.

[chrisjsewell]

I'm not sure if this partially solves your use case, but the output renderer is now "pluggable": https://myst-nb.readthedocs.io/en/latest/use/formatting_outputs.html#customise-the-render-process

[saulshanabrook]

Ah great, yeah that will let us sort of hard code in a different renderer support or try out this extensible renderer approach through that option, I believe. Thank you!

[chrisjsewell]

Ok cool, yeh let me know how you get on, and if you see any further ways it could be improved to help you out.