zarr-python icon indicating copy to clipboard operation
zarr-python copied to clipboard

document dtype extension

Open d-v-b opened this issue 6 months ago • 7 comments

This PR adds a working example of custom dtype creation and registration. because it's a lot of code, I put this in a new top-level directory called examples, which contains the executable python file dtype_example.py. This file uses PEP-723 metadata to declare a ml_dtypes dependency, and it uses a local zarr install, which means it can be tested properly against local changes.

I also expanded the current dtype docs in the user guide to include content about the data type resolution process.

TODO:

  • [ ] Add unit tests and/or doctests in docstrings
  • [ ] Add docstrings and API docs for any new/modified user-facing classes and functions
  • [ ] New/modified features documented in docs/user-guide/*.rst
  • [ ] Changes documented as a new file in changes/
  • [ ] GitHub Actions have all passed
  • [ ] Test coverage is 100% (Codecov passes)

d-v-b avatar Jun 19 '25 17:06 d-v-b

cc @nenb @ianhi, since yall were the most involved in this example over in the main dtypes PR.

d-v-b avatar Jun 19 '25 17:06 d-v-b

I think the content is basically done -- I added docstrings to basically every new class and method (please let me know if i missed anything). I am however currently stymied on some sphinx build issues, so if anyone is proficient in that kind of debugging I would appreciate help.

d-v-b avatar Jun 24 '25 14:06 d-v-b

one question for sphinx experts: how can I get sphinx to not mangle the pipe symbol (|), e.g.:

image

d-v-b avatar Jun 24 '25 15:06 d-v-b

Build issue is somehow related to this i think:

https://github.com/sphinx-doc/sphinx/issues/12026

https://github.com/readthedocs/sphinx-autoapi/issues/422

ianhi avatar Jun 24 '25 15:06 ianhi

Broken in a new way! Crazy that that fixed that

ianhi avatar Jun 24 '25 16:06 ianhi

all the sphinx issues are sorted, the only failing check is coverage, and I'm not sure I believe the claims that these changes reduced test coverage by 20%.

d-v-b avatar Jun 24 '25 18:06 d-v-b

@d-v-b FYI I think that some of these missing features in 3.0 in the array docs can now instead be linked to the new dtypes docs - https://zarr.readthedocs.io/en/stable/user-guide/arrays.html#missing-features-in-3-0.

maxrjones avatar Jun 29 '25 00:06 maxrjones

@d-v-b FYI I think that some of these missing features in 3.0 in the array docs can now instead be linked to the new dtypes docs - https://zarr.readthedocs.io/en/stable/user-guide/arrays.html#missing-features-in-3-0.

I removed the "this stuff is missing" sections, leaving only the part that is still missing (the copying + migrating data stuff)

d-v-b avatar Jul 02 '25 11:07 d-v-b