Enable Jupyter Sphinx Extension in docs?

[Peque]

While writing documentation, I noticed the extended use of the IPython Sphinx Directive.

I bet most pvlib users like to play with pvlib in interactive environments. However, I think Jupyter Notebooks are getting more common/widespread when compared to IPython consoles. I wondered if it would be acceptable to allow code in the documentation to look more like notebooks instead of consoles.

There is this extension called Jupyter Sphinx, under the umbrella of the Jupyter organization.

I think it could have a couple of advantages:

• Simpler code (i.e.: no need to @savefig)
• Removal of In [xx]:, which makes it harder to read, copy and paste
• Easier to read, since blank lines are honored in the code
• More similarities to a Jupyter notebook (so maybe more user-friendly for readers)
• Allows to hide input/output
• Allows to download as a Notebook or Python script
• Allows to emphasize lines in a cell (very useful when wanting to highlight something in the documentation)
• The style can be tweaked to, for example, remove the box shadows (to avoid changing too much the current looks of the docs)
• Allows for interactive plots with Plotly, and Pandas has a Plotly backend, so it would be very easy to have interactive plots in the documentation without adding clutter nor assuming readers need to know about Plotly/Matplotlib

A couple of (maybe) disadvantages:

• Not sure PDF (i.e.: LaTeX) export would work when using interactive plots
• ipywidgets would definitely be not rendered in PDF (https://github.com/jupyter/jupyter-sphinx/issues/61)
• ?? does not work (only used in modelchain.rst); see https://github.com/jupyter/nbclient/issues/196 (maybe replaceable by sphinx-code-include)
• Add more development dependencies (if both IPython and Jupyter were to be allowed)
• Required migration (if Jupyter was to replace IPython completely)
• Performance may be slower?

[kandersolar]

Thanks for the thorough summary @Peque! Just to have something to look at, here's the result of naively replacing ipython w/ jupyter-execute:

• jupyter-execute: https://pvlib-python--1380.org.readthedocs.build/en/1380/user_guide/modelchain.html
• ipython: https://pvlib-python.readthedocs.io/en/latest/user_guide/modelchain.html

Of course blocks of code with multiple outputs only show the last output now, so a little restructuring would be needed. Also, it seems like ?? doesn't work?

[Peque]

@kanderso-nrel Yeah, it seems Jupyter does not show ??'s output bellow the code cell, but rather at the bottom of the page:

If the IPython directive was to be replaced I agree some restructuring would be needed. Either by splitting code into multiple cells or by using print() if you wanted to show multiple outputs at the end of the cell.

I also think some CSS adjustments would be necessary in order to make it look better (mostly margins).

[Peque]

By the way, it seems ?? is only used in user_guide/modelchain.rst.

It looks like it is used to illustrate how the functions/methods are implemented (i.e.: to show the source code of the function). @wholmgren was that the intention?

If so, maybe sphinx-code-include could be used instead, with the advantage of displaying the source code with syntax highlighting (i.e.: easier to read).

[wholmgren]

It looks like it is used to illustrate how the functions/methods are implemented (i.e.: to show the source code of the function). @wholmgren was that the intention?

Yes.

Have the jupyter/ipython developers provided any guidance on the future of either tool?

[Peque]

@wholmgren What do you mean by that? Maybe we can ask them. :blush:

[wholmgren]

My impression is that ipython directive was largely abandoned for at least a few years as development focused on notebooks and then jupyter lab. It's been a little finicky. Maybe this jupyter extension will be more reliable?

[Peque]

@wholmgren It's true that the directive had 3 commits in the last 2 years, and the documentation still states:

The IPython Sphinx Directive is in ‘beta’ and currently under active development. Improvements to the code or documentation are welcome!

jupyter-sphinx seems to be a bit more active and there does not seem to be a "beta" statement in the docs.

PS: Cross-referencing https://github.com/jupyter/jupyter-sphinx/issues/36.

[bt-]

@Peque and @kandersolar, might be worth looking at nbshpinx. I use it in pvcaptest to create the examples, which are just ipynb files. I think the ability to setup mybinder to run these examples is a very nice bonus to this approach.

[kandersolar]

I've used nbsphinx a fair bit for projects that use ipynb files with sphinx (e.g. the pvlib tutorials, the engagement tracker, the unofficial psm3 user guide, and the RdTools gallery). It has worked well for me for the most part.

However I am not enthusiastic about the source for these pvlib docs pages being in .ipynb format; ipynb files are annoying to collaborate on in git/github, even after recent github improvements. I put up with it for those other projects because some unique feature of ipynb files (e.g. storing pre-calculated results) was needed but I don't think that's the case here. Having "launch on binder/colab/whatever" buttons would definitely be nice but I'm not sure it's worth switching to ipynb just for that.