jupyter-book
jupyter-book copied to clipboard
executablebooks
Major documentation sections should roughly map onto major sphinx extensions
Context
Currently we have different aspects of our documentation intermingled with one another, and grouped in very high-level categories (like "content" and "interactivity" etc). However, we might be able to define a more natural grouping of content and make it more maintainable if we followed a similar proposal as:
- https://github.com/executablebooks/jupyter-book/issues/1728
Proposal
We should explore having top-level documentation sections map on to major extensions in Jupyter Book. Probably the three most obvious ones are something like:
- MyST Parser (e.g.
Content with MyST) - MyST NB (e.g.,
Notebooks and execution) - Sphinx Book Theme (e.g.,
Layout and theming)
This might also be a good way to learn where we should move functionality from one package to another (e.g., the "launch button" functionality might make more sense in myst-nb instead of the book theme.
Tasks and updates
No response
I'm a huge :+1: on this. It's been the hardest part of learning JB - there's lots of good documentation, but it's not always clear which component is the one that it actually refers to.
I'd be interested in having the jupyter-book library become even more lightweight by dropping the configuration handling (at least, dropping the sphinx extension configuration handling e.g. myst-nb). Then the JB documentation can focus on explaining how the different parts all fit together. I feel it would be useful to users to think of JB as a nice frontend to sphinx, and understand that most of the components are just sphinx extensions.
We could still have a high-level overview, but remove most of the detailed content into the various subprojects.
I'd also like to suggest that we create a diagram e.g. with Mermaid to explain how the various components all link together.
The tricky thing is that I think we don't want an average jupyter book user to have to learn anything about Sphinx in order to use and understand jupyter book. (At least not as a bottleneck to using it)
I worry that if we have lots of deep connections to Sphinx docs, callouts to other extension websites, etc, we are going to be exposing people unfamiliar with sphinx to a lot of complexity. They'd start coming across rST and the use of conf.py for example, which i think could get really confusing.
I'm still not sure the right way to balance between these two extremes, because I agree that it would be very helpful to delegate explanation etc to each component. 🤔
Mermaid chain would be great! I think @mmcky was actually working on something like that a while back
The tricky thing is that I think we don't want an average jupyter book user to have to learn anything about Sphinx in order to use and understand jupyter book.
Yep I was going to note something similar 👍
I'm late to the JB game, so excuse me if I am rehashing old conversations. Also, this comment grew past the original scope, so let me know if I should move to somewhere more on topic,
As I mentioned earlier, my confusion with the status quo is that the JB website presents itself as the definitive guide to JB, but then there are differences between JB and the subproject documentation e.g. MyST-NB. That's not surprising at all - it's a lot of work to prune and update docs, and as an aside, I do think that the JB ecosystem has fantastic documentation.
Solving this issue will resolve this by making it clear "who owns what".
The tricky thing is that I think we don't want an average jupyter book user to have to learn anything about Sphinx in order to use and understand jupyter book.
My experience as a user has been that I've spent a reasonable amount of time searching through multiple GitHub repositories to work out what JB is actually doing. Partly this might be because sometimes docs disagree / something doesn't work, and then the user needs to understand how things fit together in order to understand why e.g. by looking at the correct repo's source code. In a perfect world, we'd just never let this happen to the docs and my point would be invalid, but I don't think that's realistic for any big, fast-moving project like this.
Therefore, I think trying to fully isolate the user from the architecture of JB is a law of diminishing returns. I think we'd benefit from a clearer page outlining how the JB jigsaw puzzle fits together. @chrisjsewell linked me to an existing page, but I think that might be too many layers of indirection away. It might be nice to have this as part of the Reference section.
However, if our aim is to ensure that the average user can get started without learning four tools, I think that's completely reasonable.
Right now, I think there are two kinds of extension that we need to consider: EB authored extensions, and external sphinx extensions. My suggestion is that the JB documentation becomes predominantly a "signpost" to the canonical docs for the JB projects because we control those and can backlink to the JB docs where necessary. For non-EB projects, I think we would maintain sections of the JB documentation for these projects to ensure that things feel consistent, with additional links to Sphinx docs / other projects where it might be helpful.
E.g.
Jupyter Book comprises of several tools that work together to build beautiful, publication-quality books and documents from computational content
Authoring Content
I tend to find the JB docs a little overwhelming, so I looked at what would happen if we:
- Removed the sections that are already found in some form in MyST-NB / MyST-Parser, i.e. the
Write ... contentsections, and theMyST syntax cheat sheet - Moved the Sphinx usage to "Advanced Jupyter Book Usage"
Obviously we want to tell users how to author content, so we'd probably need to restore a "Write Rich Content" section that links to MystNB etc as outlined above.
I agree with all of your pain points here...I like the idea of shortening the number of top-level sections, I think that could really help. Though I am still hesitant to start farming out major parts of the JB workflow to sphinx extension sites, again in my opinion, 80% of the Jupyter Book users should never have to see the word "Sphinx".
One reason you're hitting some pain is because there is a large gap between Jupyter Book's current functionality and the MyST-NB and MyST-parser latest releases. This is because we're operating on limited resources and it takes time to propagate updates up into Jupyter Book, update the docs, etc. I hope that over time as those packages become more stable, this will become less of a pain point, tough I appreciate that this is not helpful to you since you need to use it now not in the future :-)
Though I am still hesitant to start farming out major parts of the JB workflow to sphinx extension sites
To clarify, I'm only proposing linking to the Executable Books projects (e.g. MystNB). These can have a landing page for JB if neccessary, and would focus on usage e.g. syntax. Where we don't have the ability to do this, e.g. sphinxcontrib-bibtex, we would link to our own maintained (internal) sections within JB.
One reason you're hitting some pain is because there is a large gap between Jupyter Book's current functionality and the MyST-NB and MyST-parser latest releases
Certainly this is a phenomenon too, and that's the nature of move fast and break things! Though if we link to the versioned documentation from Jupyter Book, I think it'd be harder to accidentally find new syntax.
Not to distract too much from the original issue (really like the idea of trying to clarify the various dependencies/components and simplify docs navigation)! But:
Though if we link to the versioned documentation from Jupyter Book, I think it'd be harder to accidentally find new syntax.
I think this is a great idea! I've run into this a number of times, most recently with learning about the connection between myst-nb and jupyterbook. The JB docs here https://jupyterbook.org/en/stable/file-types/myst-notebooks.html point to https://myst-nb.readthedocs.io/en/latest/index.html. But JB pins myst-nb to <0.14 https://github.com/executablebooks/jupyter-book/blob/efc954744b9be62fa24ceed1ba34f9ed0b8108d0/pyproject.toml#L36
The 'latest' myst-nb release is now 0.16 with seemingly a lot of new functionality :( So if explicit dependency pins are being used just changing links to the versioned docs would be helpful! (https://myst-nb.readthedocs.io/en/v0.13.2)
