[RFC] Use mkdocs-material for Zarr-Python documentation

[maxrjones]

This PR switches the docs from sphinx to mkdocs-material (closes #2894)

Preview: https://zarr--3118.org.readthedocs.build/en/3118/

Motivation for using mkdocs-material

• Markdown-first: Uses standard Markdown instead of reStructuredText, which is more widely known and easier to write
• Simpler configuration: Single YAML config file vs. complex Python configuration
• Lower learning curve: Much easier for new contributors to get started writing documentation
• Active development: Very actively maintained with frequent updates
• Growing ecosystem: Large community and extensive plugin ecosystem
• Modern UI/UX: Clean, professional design that feels contemporary
• More navigable API docs: The customization of the API reference pages makes classes and functions easier to find (this was my main problem with the existing doc's usability).

Changes

Apart from the conversion to markdown and configuration updates, I made the following changes:

• Remove talks/scipy2019/submission since it wasn't included in the docs
• Consolidate quickstart with the home page
• Consolidate about with the home page
• Remove roadmap since it's purpose has been completed

Review request context

I'm not looking for a final review here, but rather a general consensus on whether it's worth spending the time that'd be needed for the last 20% of polish.

TODO

• [ ] Update any syntax in docstrings
• [ ] Update doctests
• [ ] Changes documented as a new file in changes/
• [ ] GitHub Actions have all passed
• [x] Setup versioning
• [x] Add redirects

[d-v-b]

I love the way this looks! Thanks so much for working on this.

[d-v-b]

@zarr-developers/python-core-devs it would be good to get some feedback on this. Switching our docs is a big decision, and it only makes sense to put effort in this if everyone is aligned. I personally think we should 100% move to mkdocs-material, but it would be good to hear if there are any big objections to that.

Aesthetically, and in terms of user-friendliness, I think this PR represents a huge improvement. If we just consider the landing page, this:

is vastly better than this:

[joshmoore]

Apologies, I thought I had :+1:'d previously. Now doing that retroactively in a general sense of the move and specifically for the style & the use of markdown.

But I agree that we need a careful review of the pages (and personally, I liked the cards on the landing page)

[sanketverma1704]

Thanks a lot for working on this, @maxrjones! Looks great!

I love the switch from .rst to .markdown—it makes it easy to update and lowers the barrier for contribution.

I liked the cards on the landing page

I'm with Josh on this one. Can we also add cards in mk-docs?

[d-v-b]

i am actually strongly anti-cards, because they take up a huge amount of space that could be devoted to actual content.

Here's how much space the "user guide" link consumes in the mkdocs version in this PR:

Compare with the space used in our current version for the same information:

...and why do we need text on this card that explains what a user guide is? Isn't it evident that a user guide is a guide for users?

Similarly, if we take the "quick start" card, IMO the first thing people see when they come to the docs should be the "quick start" content. We shouldn't ask people to click on a link after coming to our docs to see how to use zarr. In max's PR, he puts that content front-and-center on the main page:

by contrast, the old docs waste a huge amount of space with the card, which also has a very confusing "person escaping from an industrial accident" icon:

So yeah, in summary part of what makes this PR great to me is the removal of the cards. They are dead weight from a UX POV.

[maxrjones]

While I prefer the no-cards version, I also believe it's preferable to decide based on the ease of use for people who aren't extremely familiar with the Zarr-Python documentation, because those people may be completely blocked or dissuaded from using the library whereas others will likely find their way anyways. I could do some polling on preferences if others agree with this philosophy.

I also think that the transition to mkdocs shouldn't be blocked by whether the home page has cards. If people think this'll take a while to decide on, I could add this style card for now to minimize the stylistic decisions included in this PR.

[jhamman]

I just spent some time here and wanted to give a big 👍 to the switch to mkdocs-material. I wanted to do this leading up to 3.0 but choose not to in order to avoid scope creep in the documentation update.

As for cards, I don't think this should bend the decision here either way. We should optimize for usability. If users can quickly find their way to the main sections of the docs, then I'm for dropping the cards.

[jni]

fwiw, using only markdown in sphinx docs is easy to do these days, including automatically running executable jupytext notebooks, which we use in the napari docs. Example page:

https://github.com/napari/docs/blob/main/docs/howtos/layers/image.md?plain=1

Output:

https://napari.org/stable/howtos/layers/image.html

Never been a fan of the material look, but maybe I'm just weird. I'd prefer contributing upstream to the pydata sphinx theme (including customisation options) vs switching to a completely different ecosystem. Completely happy to be outvoted here, just wanted to add an alternative viewpoint in case there's more people like me out there. 😂

[jni]

(in short, out of all the motivations, I only find the yaml configuration compelling...)

[maxrjones]

Thank you so much to everyone who's expressed opinions so far!

I'd greatly appreciate if anyone still lurking could share your comments before July 3, 2025. The motivation for this deadline is that I'd like to make progress on this PR over the long weekend in the U.S. but only if the change to mkdocs seems likely to be accepted.

[normanrz]

I think this looks great :+1: Definitely like the markdown! Had good experiences with mkdocs previously as well.

[TomAugspurger]

+1 to all of @jni's comments (including the "Completely happy to be outvoted here" bit; ultimately, ensuring that contributors are empowered and motivated to write good documentation is the most important thing. I can adapt if others prefer different systems).

[ianhi]

I love writing docs in markdown, and I firmly believe all docs should be executed regularly. So I am +1 on this being the right direction.

That said it might be nice to in the future do a further swap to a myst markdown system (possibly with sphinx-immaterial). The reason being that for the kinds of executable docs common in scientific python ecosystem I find markdown-exec to have some important limitations.

• There is a lot of boilerplate on every code block to set up the display, session, and execution (https://github.com/pawamoy/markdown-exec/issues/76, https://github.com/pawamoy/markdown-exec/issues/77)
• There is limited ability to specify which code failures/exeception should cause a docs build failure (https://github.com/pawamoy/markdown-exec/issues/75)
  • This is problematic if you want show that a code block fails in a specific way
• No easy integration with notebook formats
• RIch outputs can be complicated (e.g. embedding matplotlib requires printing out the buffer: https://pawamoy.github.io/markdown-exec/gallery/#exec-4--source)

the myst ecosystem has a solution to all those issues which is why I generally much prefer it.

However, from talking with @maxrjones and @d-v-b it the impediment to using the the markdown options in sphinx is the inability to make clickable typealiases in docstrings (https://github.com/zarr-developers/zarr-python/issues/2289) . That is crucial functionality and clearly makes using myst/sphinx not an option right now. I have a (maybe?) fix for that in sphinx: https://github.com/sphinx-doc/sphinx/pull/13808 but in the meantime this PR is nice improvement.

[codecov[bot]]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 61.21%. Comparing base (ef136e9) to head (9404483). :warning: Report is 56 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3118   +/-   ##
=======================================
  Coverage   61.21%   61.21%           
=======================================
  Files          84       84           
  Lines        9923     9923           
=======================================
  Hits         6074     6074           
  Misses       3849     3849           

Files with missing lines	Coverage Δ
src/zarr/abc/store.py	38.29% <ø> (ø)
src/zarr/api/asynchronous.py	72.20% <ø> (ø)
src/zarr/api/synchronous.py	36.61% <ø> (ø)
src/zarr/codecs/numcodecs/_codecs.py	38.46% <ø> (ø)
src/zarr/convenience.py	0.00% <ø> (ø)
src/zarr/core/array.py	68.62% <ø> (ø)
src/zarr/core/attributes.py	46.15% <ø> (ø)
src/zarr/core/buffer/core.py	30.98% <ø> (ø)
src/zarr/core/buffer/cpu.py	38.18% <ø> (ø)
src/zarr/core/config.py	29.16% <ø> (ø)
... and 12 more

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[maxrjones]

A status update:

• The conversion of the landing page, quick-start, user-guide, developer guide, and release notes are complete
• The API documentation has been structured for mkdocs
• The remaining task is to update reference styles in the docstrings.

I appreciate all the high-level feedback on the switch. Thank you @ianhi in particular for discussing this PR the Zarr dev meeting and for summarizing your perspective on sphinx vs mkdocs above. I'm actually about to head out for a vacation until Monday, so if you or anyone else wants to use the completed rst->md conversion with a sphinx/myst based setup it'd be a good time to do so. It might be obvious that I prefer mkdocs over sphinx, but I respect that each have their strengths. Mostly, I just find the current API documentation really hard to use and want something better. I'm willing to spend the time on this massive update, but if someone else wants to finish the work using sphinx I certainly wouldn't complain 😉

[maxrjones]

@zarr-developers/python-core-devs this PR is now ready for review. I recommend reviewing it without striving for perfection because many people share the opinion that the current documentation is difficult to use, merging would allow other people to also contribute to the docs improvements, and each additional merge commit increases the chances of new mistakes creeping in. Thanks for your consideration!

[TomNicholas]

It looks beautiful. There are some small things that could be improved, but I think you should merge.

Thing that could be improved: The formatting of code examples doesn't make it clear which cells are executed and which are results:

[d-v-b]

thank you @maxrjones!