cubed icon indicating copy to clipboard operation
cubed copied to clipboard

Test with Zarr V3

Open tomwhite opened this issue 2 years ago • 1 comments

Add a GH Actions workflow that runs Cubed tests using Zarr V3 storage. This will ensure that the upcoming work to update Zarr to support the V3 spec (see https://github.com/zarr-developers/zarr-python/discussions/1480) is working in Cubed.

I've done a preliminary test in https://github.com/tomwhite/cubed/tree/zarr-v3-testing, and the current V3 implementation works with Cubed. (Run with ZARR_V3_EXPERIMENTAL_API=1 pytest -vs)

The next step is to turn this into a workflow that runs against Zarr's V3 branch (when it is created).

cc @MSanKeys963 @jhamman

tomwhite avatar Aug 22 '23 16:08 tomwhite

Zarr 3.0.0a0 was released last week. It's on PyPI, and can be installed by specifying --pre with pip install.

There's already a GH Actions workflow for this, which is currently failing (e.g. here's a recent run).

Some useful pre-requisites to getting Zarr V3 working are:

  • [x] #479
    • (Removes a dependency on an internal Zarr API)
  • [x] #441
    • (Removes a dependency on BasicIndexer, another internal Zarr API)
  • [x] #481
    • (Removes more dependencies on internal Zarr indexing APIs)
  • [x] #480
    • (Adds a mechanism to try out Zarr V3)

The following bugfixes are needed in Zarr:

  • [x] https://github.com/zarr-developers/zarr-python/issues/1975
  • [x] https://github.com/zarr-developers/zarr-python/issues/1976
  • [ ] https://github.com/zarr-developers/zarr-python/issues/1977
  • [x] https://github.com/zarr-developers/zarr-python/issues/1978

tomwhite avatar Jun 17 '24 15:06 tomwhite