zarr-developers
Issue with xarray and zarr when dimensions start with "meta"
Zarr version
2.12.0
Numcodecs version
0.10.1
Python Version
3.8.10, 3.10.5
Operating System
Linux
Installation
pip
Description
Currently I'm working with xarray for MRSI analysis. I have been using xarray datasets with one the dimensions labeled as "metabolite." Previously I have been able to save and load the data to zarr with no issues using xr.Dataset.to_zarr and xr.open_zarr.
Currently I'm getting an issue where I get an error raised by zarr that complains about this dimension starting with "meta." I think this may be due to a new version of zarr. I have copied the error below.
When I changed the dimension name (i.e. the zarr subfolder and .zmetadata file) from "metabolite" to something that doesn't start with "meta" then I can load the data properly.
Someone may need to modify the xr.Dataset.to_zarr and xr.open_zarr functions in case an xarray user decides to create a dimension that starts with "meta" and wants to save their dataset to zarr.
Please let me know if you need to send any additional info.
Steps to reproduce
See description. Here's a link to the bug report I posted on xarray's github page: https://github.com/pydata/xarray/issues/6853
Additional output
No response
Thanks, @zndr27. I assume there needs to be a separator included in:
assert not path.startswith("meta")
like "meta/". cc: @grlee77
We also saw this issue because we had our internal "metadata" group. Currently, we limited the upper bound for the zarr dependency to <2.12 until there is a solution.
What are the reserved prefixes for the new v3 spec that users should be aware of? Maybe this also warrants a documentation piece as well?
Yes, I can make a PR to allow names like metabolite. The intention was not to prevent these, so I think we just need to fix it in the next patch release.
Just wanted to check if your pull request has been accepted. When is the next patch release supposed to happen?
Hi @zndr27, thanks for checking in. I just made a PR with a fix today (#1123). I think a v2.13 release is planned to happen fairly soon, so hopefully it will make it in for that.