Numcodecs in v3

[brokkoli71]

Implements numcodec_to_zarr3_codec to enable numcodec support in zarr v3 arrays This PR is a result of moving the to_zarr3 logic from https://github.com/zarr-developers/numcodecs/pull/741 to zarr-python

• fixes #2964

TODO:

• [x] Add unit tests and/or doctests in docstrings
• [ ] Add docstrings and API docs for any new/modified user-facing classes and functions
• [ ] New/modified features documented in docs/user-guide/*.rst
• [ ] Changes documented as a new file in changes/
• [ ] GitHub Actions have all passed
• [ ] Test coverage is 100% (Codecov passes)

[d-v-b]

it might be useful to have a conversation about the strategy here. It looks like numcodecs_to_zarr_v3_codec dynamically creates new classes. My preference would be to avoid any dynamically created classes, since this makes the codebase harder to document / reason about, and can lead to very confusing bugs. I think it's easier in the long term if we explicitly define each codec.

[brokkoli71]

@d-v-b Are you thinking of something like https://github.com/zarr-developers/numcodecs/blob/main/numcodecs/zarr3.py or even moving all the logic from this file to zarr-python?

[d-v-b]

@d-v-b Are you thinking of something like https://github.com/zarr-developers/numcodecs/blob/main/numcodecs/zarr3.py or even moving all the logic from this file to zarr-python?

the file you linked also contains dynamic class generation, which was problematic and motivated this PR: https://github.com/zarr-developers/numcodecs/pull/745, which removes the dynamism.

and yes I think that all of this stuff should live in zarr-python.

[brokkoli71]

I changed numcodec_to_zarr3_codec to use registered numcodecs.zarr3-Codecs instead. Now that https://github.com/zarr-developers/numcodecs/pull/745 has been merged, there should be no dynamic class generation, correct? What do you think of the current approach, @d-v-b? Feel free to give suggestions.