zarr-python icon indicating copy to clipboard operation
zarr-python copied to clipboard

codec x dtype interactions

Open d-v-b opened this issue 3 months ago • 0 comments

in #3332 I'm building out zarr v3 support for codecs that perform a data type transformation, namely Astype, FixedScaleOffset and Delta.

All of these codecs contain a data type specification in their configuration. At the moment, that representation is a Zarr v2 / numpy string like "|u1" which is totally out of place in Zarr v3, which introduced a new, incompatible data type representation.

The changes in #3332 will include logic that consistently uses Zarr V2 data type representations when serializing codec configs in a zarr v2 context, and Zarr V3 representations in a zarr v3 context, but people should generally be aware that there's a tight coupling between data types and codecs that we were not properly handling before.

d-v-b avatar Sep 25 '25 16:09 d-v-b