zarr-python
zarr-python copied to clipboard
zarr-developers
.zmetadata clarification
re: https://github.com/zarr-developers/zarr-python/pull/268
The .zmetadata object is needed to hold the consolidated meta-data. But as far as I can tell, the V2 spec does not mention this even as optional. Can someone point me to some revised version of the V2 spec that describes .zmetadata?
@DennisHeimbigner: .zmetadata is currently outside of the v2 spec (i.e. an implementation detail) like dimension_separator was.
@joshmoore
I have a question regarding this. So when we consolidate metadata with / dimension separator, it ends up writing the metadata file as zmetadata instead of .zmetadata; and the open_consolidated fails by default.
CORRECTION: approach 1 below doesn't work. It has to be the latter; but then the zmetadata file is not hidden and also users have to specify it every time they open it with open_consolidated.
We can alleviate this by consolidating metadata with a store that still has . as separator; or by providing the zmetadata name to open_consolidated. However, either solution is not very elegant.
What could be a possible solution to this? Should we wait until this becomes a standard in v3 for an elegant solution or is there some low-effort solution with current v2?
Thanks!
is there some low-effort solution with current v2?
I think so, yes. This sounds very much like a straight-forward bug (thanks for opening as #1121). Somewhere there's a check for "." which needs to be made smarter.
