zarr-specs icon indicating copy to clipboard operation
zarr-specs copied to clipboard

Non-JSON metadata and attributes

Open axtimwalde opened this issue 5 years ago • 11 comments

As briefly discussed in the group chat, I would like to propose a change to how metadata and attributes are accessed. The current spec is specific that this data must be readable and writable as JSON. This is compatible with all current storage backends of Zarr and the filesystem and cloud storage backends of N5. It is not compatible with the current HDF5 backend of N5 where attributes and metadata are represented as HDF5 attributes. Instead of requiring JSON, I suggest that metadata and attribute access should be specified similar to the group and array access protocol of the spec, i.e. as access primitives, i.e. API. The most basic primitives would be:

getAttribute - Retrieve the value associated with a given key and attributeKey.

| Parameters: `key`, `attributeKey`, [`type`]
| Output: `value`

setAttribute - Store a (key, attributeKey, value) triple.

| Parameters: `key`, `attributeKey`, `value`
| Output: none

Probably also something to list attributes and may be infer their types if necessary. The N5 API does it this way and I find it very straight forward to use this across JSON and non-JSON backends

https://github.com/saalfeldlab/n5/blob/master/src/main/java/org/janelia/saalfeldlab/n5/N5Reader.java#L214

https://github.com/saalfeldlab/n5/blob/master/src/main/java/org/janelia/saalfeldlab/n5/N5Reader.java#L271

https://github.com/saalfeldlab/n5/blob/master/src/main/java/org/janelia/saalfeldlab/n5/N5Writer.java#L43

https://github.com/saalfeldlab/n5/blob/master/src/main/java/org/janelia/saalfeldlab/n5/N5Writer.java#L59

and the default JSON implementation which is only bloated to support version 0 with non auto-inferred compressors

https://github.com/saalfeldlab/n5/blob/master/src/main/java/org/janelia/saalfeldlab/n5/AbstractGsonReader.java

axtimwalde avatar May 23 '19 21:05 axtimwalde