zarrs-python
zarrs-python copied to clipboard
Fix unsound assumption of chunks being disjoint
See https://github.com/ilan-gold/zarrs-python/pull/87#discussion_r1966169606
I assume we could still do things in parallel if we want.
We could create a DisjointViews (or so) struct that holds non-overlapping ArrayBytesFixedDisjointViews and has a safe fn add(&mut self, ...) -> bool that returns if a region was safely added (would not overlap with the already contained regions) or not. Then at runtime,
- create a
Vec<DisjointViews>by first trying to add each view to the firstDisjointViewsobject, then the second, … until each view has been added. - go over the vec and handle each
DisjointViewsobject in parallel.
There could be pathological cases where that doesn’t work well, but in practice, those shouldn’t happen
That sounds reasonable, but we could just be patient. zarr-python might disallow it https://github.com/zarr-developers/zarr-python/issues/2855