zarr-python icon indicating copy to clipboard operation
zarr-python copied to clipboard

Adding Quantize filter to ds.to_zarr encoding does not work

Open raphaeljolivet opened this issue 6 months ago • 1 comments

Zarr version

3.0.8

Numcodecs version

0.16.1

Python Version

3.12

Operating System

Linux

Installation

pip + venv

Description

I am struggling to add a Quantize filter before compression, with zarr v3. I work with xarray (2025.6.1)

I get the following error : TypeError: Expected a ArrayArrayCodec. Got <class 'numcodecs.quantize.Quantize'> instead.

Using from numcodecs.zarr3 import Quantize leads another error : TypeError: Object of type type is not JSON serializable

What is the proper way to add filters to encoding with zarr V3 & *xarray ?

Steps to reproduce

from numcodecs import Quantize
import numpy as np

encoding={
   "var" : dict(filters=[Quantize(digits=2, dtype=np.float32)])
}
ds.to_zarr(..., encoding=encoding)

Additional output

No response

raphaeljolivet avatar Jun 24 '25 10:06 raphaeljolivet

@raphaeljolivet thanks for this report. I think you have found 2 problems, both of which need to be fixed:

  • our codecs code needs to be simplified, so that users can bring in numcodecs codecs without first converting them to the new zarr v3 codecs.
  • there's a bug in the zarr v3 version of the numcodecs quantize

d-v-b avatar Jun 26 '25 07:06 d-v-b