Appending to a ZARR dataset removes dimension separator attribute

[jbeezley]

What happened?

When creating a ZARR store with dimension_separator="/", the to_zarr_store method removes dimension_separator attribute in the .zarray file.

What did you expect to happen?

It should retain the original storage attributes.

Minimal Complete Verifiable Example

import json

import xarray as xr
from zarr.storage import MemoryStore


store = MemoryStore(dimension_separator="/")

ds = xr.Dataset(
    {"data": xr.DataArray([0.0], dims=("x",))},
    coords={
        "x": xr.DataArray([0.0], dims=("x",)),
    },
)
ds.to_zarr(store, mode="w", consolidated=False)
assert json.loads(store["data/.zarray"].decode()).get("dimension_separator") == "/"

ds = xr.Dataset(
    {"data": xr.DataArray([1.0], dims=("x",))},
    coords={
        "x": xr.DataArray([1.0], dims=("x",)),
    },
)
ds.to_zarr(
    store,
    append_dim="x",
    consolidated=False
)
assert json.loads(store["data/.zarray"].decode()).get("dimension_separator") == "/"

MVCE confirmation

• [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• [X] Complete example — the example is self-contained, including all data and the text of any traceback.
• [X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• [X] New issue — a search of GitHub Issues suggests this is not a duplicate.
• [X] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

AssertionError                            Traceback (most recent call last)
File ~/xarray_repro/run.py:29
     18 ds = xr.Dataset(
     19     {"data": xr.DataArray([1.0], dims=("x",))},
     20     coords={
     21         "x": xr.DataArray([1.0], dims=("x",)),
     22     },
     23 )
     24 ds.to_zarr(
     25     store,
     26     append_dim="x",
     27     consolidated=False
     28 )
---> 29 assert json.loads(store["data/.zarray"].decode()).get("dimension_separator") == "/"

AssertionError:

Anything else we need to know?

No response

Environment

INSTALLED VERSIONS ------------------ commit: None python: 3.11.5 (main, Aug 24 2023, 15:18:16) [Clang 14.0.3 (clang-1403.0.22.14.1)] python-bits: 64 OS: Darwin OS-release: 22.5.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: None libnetcdf: None

xarray: 2023.11.0 pandas: 2.1.3 numpy: 1.26.2 scipy: None netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.16.1 cftime: None nc_time_axis: None iris: None bottleneck: None dask: 2023.11.0 distributed: None matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2023.12.0 cupy: None pint: None sparse: None flox: None numpy_groupies: None setuptools: 68.1.2 pip: 23.2.1 conda: None pytest: None mypy: None IPython: 8.18.1 sphinx: None

[welcome[bot]]

Thanks for opening your first issue here at xarray! Be sure to follow the issue template! If you have an idea for a solution, we would really welcome a Pull Request with proposed changes. See the Contributing Guide for more. It may take us a while to respond here, but we really value your contribution. Contributors like you help make xarray better. Thank you!

[max-sixty]

When creating a ZARR store with dimension_separator="/", the to_zarr_store method removes dimension_separator attribute in the .zarray file.

Is it that appending removes the dimension_separator? (there's no to_zarr_store method...)