Tobias Kölling

:+1: for moving in small steps. And I also agree to the rest of the post @madsbk The only thing vision-wise which I believe should be accounted for right now,...

I'm a bit late in the discussion, but hope I may still join in. Generally, I really like the idea of sharding, and I believe that any of the possible...

@jstriebel thanks for getting back. I think my thought maybe better described as a little bit of reframing which probably results in making it more cross-topic. But in the end,...

@normanrz if we'd use only plain morton codes, yes, we would only get cube shaped shards. I'd expect that in many cases, this is what one would actually want to...

I'm trying to wrap my head around what the **pros** and **cons** of the Store and Array based prototypes are and how they might interact with other topics (like checksums,...

Thanks @jstriebel for these awesome drawings! So the issue with checksums is, I believe it's not possible to do partial checks. Thus, if someone would try to do a `partial_getitem(some_offset)`...

Yes, checksums could in theory be implemented either on shard or on chunk level and independent of the store. But the choice has practical implications (independent of concrete checksumming or...

Thanks @rabernat for pointing me here. I'm thinking a bit around this and our little initial discussion about [IPLD](https://ipld.io/docs/). After reading this thread, I believe that IPLD might indeed be...

Oh... and given someone has written out the Merkle tree structure in the form of IPLD including `.zarray`-Metadata, like: ``` { ".zarray": {"chunks": [...], "compressor": {...}, "dtype": "...", ...}, "0.0":...

> One complexity here is that there are two possible hashes for each chunk: I'd say yes and no. If we'd make the way of compressing the data a parameter...