Jeremy Maitin-Shepard
Jeremy Maitin-Shepard
First of all, the spec version will probably have to be bumped to version 3, since it is an incompatible change --- version 2 implementations that attempt to read the...
As far as Morton order, whether it is advantageous depends on the access pattern. There are also many variations possible, e.g. how the dimensions are ordered, whether some dimensions should...
I agree with @chris-allan that zarr has focused on simplicity of the data format and implementation, and as that has served a class of use cases well it may well...
It would be great if the zarr specification owners weigh in on this. I think a sharded format would be tremendously useful and it would be great to have it...
Great to see that consensus is building around "format 2". Regarding the description by @jstriebel of the format in the pull request, I would suggest a couple minor changes: -...
@chris-allan Re: conditional reads and writes: Yes, I am referring to the `ifGenerationMatch` and similar options. I was not proposing that zarr-python necessarily make use of optimistic concurrency, merely that...
@chris-allan I think the issue of "misbehaving clients" is not too severe, because at least in the most common parallel writing use cases I would imagine (distributed processing pipelines), all...
Thank you @joshmoore for your comments: > Though there does seem to be general support for the v2 format, the foremost goal of this initial effort is to find the...
@mkitti What do you see as the advantage of making the shard file also a valid HDF5 file? That would also require making the zarr compressors and filters available to...
Making the root group the default seems like a reasonable choice, but I think it would be nice to more generally be able to specify a zarr group or array...