pyxem
Replace nbsphinx tutorials with sphinx-gallery examples
I'd like us to replace the nbsphinx tutorials (notebooks) with sphinx-gallery examples, because:
- any gallery example where some functionality is explicitly used is listed at the bottom of the technical reference
- example scripts are easier to maintain than the tutorials
- example scripts are therefore also easier for new contributors to contribute
- sphinx-gallery is more widely used (and maintained) than nbsphinx
One benefit with nbsphinx is the ability to cache notebook output and therefore avoid having to actually run the code on building the docs. However, since we don't use this currently, I think it's fine to lose that option.
Another drawback is that links to tutorials will be lost. We could try to set up redirecting on Read the Docs, but I'm not sure it's worth it.
Any objections or thoughts?
@hakonanes I love this idea. Considering the docs already take ~15 min to build on RTD I'm a little concerned that you are close to getting your jobs killed there with 1-2 more examples. Although that might happen no matter what way you take. The 15 min cap for community jobs isn't a hard cap rather seems to be a soft cap depending on the available resources. In any case you might eventually have to change to what has been discussed in https://github.com/pyxem/pyxem/issues/1150. It might be worthwhile to think about a different website name so that if you need to switch from RTD it should be easier on people. Seems like orix.org is currently taken... You could probably use pyxem.org/orix if you think that might be useful?
I don't think the redirecting is probably worth it. Although I think that you can have two separate directories. \Examples and \Tutorials and then just keep the links etc.
Considering the docs already take ~15 min to build on RTD I'm a little concerned that you are close to getting your jobs killed there with 1-2 more examples
That's a fair point. Looking at our build logs, they do go slightly over 15 minutes.
I'm not opposed to your suggestions, but we should try to make the RTD build faster before looking at other hosting options. I replacing tutorials will save some time... We'll see. I'll also look at speeding up the technical reference creation.
Yea I'd avoid it as long as possible as it's not terribly fun to do...
The main thing that https://github.com/pyxem/pyxem/issues/1150 does that really helps is caches the downloading of zenodo files which can be slow at times and make it hard to reproduce things.
I wanted to add all of https://www.pyxem.org/en/stable/tutorials/index.html to the examples but the run time for all the tutorials is quite extensive.
I'm not as familiar as you two with the nuances of how RTD builds, but I'm fully on board with the main suggestion of doing away with notebooks for all the reasons previously discussed
Another drawback is that links to tutorials will be lost. We could try to set up redirecting on Read the Docs, but I'm not sure it's worth it.
I'm not worried. Near-duplicates of every file will still exist in the ReadTheDocs navigation tree.
Considering the docs already take ~15 min to build on RTD I'm a little concerned that you are close to getting your jobs killed there with 1-2 more examples.
It looks like every existing example compiles in under 1 second except this one, which makes me think there is some merit to @CSSFrancis and @ericpre conversation about caching and/or synthetic data. Might be worth timing the individual build steps if RTD ever does decide to error out.
There is merit to caching data instead of downloading every time. But, the IPF density plot example is slow because of the pole density calculation. It doesn't actually download the data. Data is downloaded at the start of building docs:
https://github.com/pyxem/orix/blob/7580f15503c8d9a9531a712545f636925d1328a2/doc/conf.py#L360-L364
Subsequent calls, like in the example, only load the cached data.
the IPF density plot example is slow because of the pole density calculation. It doesn't actually download the data.
Agreed, yes. Sorry, I came to the same conclusion on my own, but my comment was misleading. The build times are long, but the examples themselves do not seem to be the problem. It's worth figuring out eventually, but not strictly relevant to the idea of ditching notebooks in favor of examples, which it seems we all agree is a great move.
https://github.com/pyxem/pyxem/pull/1152 also cache the example gallery. I don't know if it would be possible to also cache the tutorial, maybe something to look at.
The tutorial showing clustering of orientations from the 2D titanium map is too heavy to be computed by Read The Docs (limited by memory and time), which is why we've stored it with cell output. I think this is OK, as long as we re-run it before we make new releases.
This notebook cannot be included in the Sphinx Gallery, as far as I can tell. So, I'll have to leave the tutorials section intact, with only one tutorial. And nbsphinx as a dependency.
We may also consider reverting to storing cell outputs with the misorientation clustering notebook, as it is also relatively memory and time intensive.
2D titanium map is too heavy ...
Just a thought, variations of this issue are going to start cropping up if we continue fleshing out EBSD features like CrystalMap.Regions or 3d datasets, which will require more memory-intensive examples to demo. With that in mind, I think it's probably a good idea to leave a few of these as jupyter notebooks.
We could also consider the idea of creating a seperate "pyxem/EBSD_tutorials" repo if we ever run out of build time for ORIX, though that seems like overkill right now...
I think you're right. But, we should try our best to use small examples that just demonstrates the functionality. Not full tutorials showing every minute detail. Otherwise, the documentation gets bloated and difficult to update.