zarr-python icon indicating copy to clipboard operation
zarr-python copied to clipboard

V2 PCodec decompression error

Open slowjazz opened this issue 6 months ago • 0 comments

Zarr version

3.0.8

Numcodecs version

0.15.1

Python Version

3.12.7

Operating System

Mac

Installation

uv pip

Description

I write an array with Pcodec and dtype of np.float32, using zarr_format=2 Using zarr-python 3 will throw if I try to read it.

Steps to reproduce

Example:


# /// script
# dependencies = [
#   "zarr>=3,<4",
#   "pcodec==0.3.4",
#   "numcodecs==0.15.1"
# ]
# ///
#

import zarr
import numpy as np
import sys
from numcodecs import PCodec

print(sys.version)
print(zarr.__version__)

# write zarr
compressor = PCodec(level=12)
z = zarr.open(
    "test.zarr", mode="w", shape=(1000, 1000), chunks=(10, 10), compressor=compressor, zarr_format=2
)
z[:] = np.ones((1000, 1000))

# open zarr
z = zarr.open('test.zarr', zarr_format=2)
z[:50] # throws
zarr.print_debug_info()

uv has a problem with uv run v3-example.py, presumably because of some non-python panic, but the stack trace after installing the reqs is:

3.12.7 (main, Oct  1 2024, 02:05:46) [Clang 15.0.0 (clang-1500.3.9.4)]
0.1.dev2553+gbbdefac
Traceback (most recent call last):
  File "/Users/kli/workplace/zarr-python/v3-example.py", line 19, in <module>
    z = zarr.open('test.zarr', zarr_format=2)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/_compat.py", line 43, in inner_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/api/synchronous.py", line 201, in open
    obj = sync(
          ^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/sync.py", line 163, in sync
    raise return_result
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/sync.py", line 119, in _runner
    return await coro
           ^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/api/asynchronous.py", line 355, in open
    return AsyncArray(store_path=store_path, metadata=_metadata_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/array.py", line 280, in __init__
    metadata_parsed = parse_array_metadata(metadata)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/array.py", line 161, in parse_array_metadata
    return ArrayV2Metadata.from_dict(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/metadata/v2.py", line 196, in from_dict
    return cls(**_data)
           ^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/metadata/v2.py", line 93, in __init__
    compressor_parsed = parse_compressor(compressor)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/metadata/v2.py", line 301, in parse_compressor
    return numcodecs.get_codec(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/numcodecs/registry.py", line 54, in get_codec
    raise UnknownCodecError(f"{codec_id!r}")
numcodecs.errors.UnknownCodecError: codec not available: ''pcodec''

Additional output

3.12.7 (main, Oct  1 2024, 02:05:46) [Clang 15.0.0 (clang-1500.3.9.4)]
3.0.8
platform: macOS-15.4.1-arm64-arm-64bit
python: 3.12.7
zarr: 3.0.8

**Required dependencies:**
packaging: 25.0
numpy: 2.3.1
numcodecs: 0.15.1
typing_extensions: 4.14.0
donfig: 0.8.1.post1

**Optional dependencies:**
numcodecs: 0.15.1

**Not Installed:**
botocore
cupy-cuda12x
fsspec
s3fs
gcsfs
universal-pathlib
rich
obstore

I've also found that creating the array with zarr version 2.18.7 results in the same outcome.

slowjazz avatar Jun 27 '25 19:06 slowjazz