zarr-python

zarr-python copied to clipboard

re: https://github.com/zarr-developers/zarr-python/pull/268

The .zmetadata object is needed to hold the consolidated meta-data. But as far as I can tell, the V2 spec does not mention this even as optional. Can someone point me to some revised version of the V2 spec that describes .zmetadata?

@DennisHeimbigner: .zmetadata is currently outside of the v2 spec (i.e. an implementation detail) like dimension_separator was.

@joshmoore

I have a question regarding this. So when we consolidate metadata with / dimension separator, it ends up writing the metadata file as zmetadata instead of .zmetadata; and the open_consolidated fails by default.

CORRECTION: approach 1 below doesn't work. It has to be the latter; but then the zmetadata file is not hidden and also users have to specify it every time they open it with open_consolidated.

We can alleviate this by consolidating metadata with a store that still has . as separator; or by providing the zmetadata name to open_consolidated. However, either solution is not very elegant.

What could be a possible solution to this? Should we wait until this becomes a standard in v3 for an elegant solution or is there some low-effort solution with current v2?

Thanks!

is there some low-effort solution with current v2?

I think so, yes. This sounds very much like a straight-forward bug (thanks for opening as #1121). Somewhere there's a check for "." which needs to be made smarter.

closing this now and cross referencing with the v3 todo list here: #1161