z5 icon indicating copy to clipboard operation
z5 copied to clipboard

(slightly more) complex dataset indexing

Open clbarnes opened this issue 5 years ago • 1 comments

Inspired by z5py and with an eye on libraries doing similar things, I wrote https://github.com/clbarnes/h5py_like , which has a bunch of abstract base classes for Files, Groups and Datasets. An example where it wraps rust-n5 is here: https://github.com/clbarnes/rust-pyn5/tree/h5pylike/pyn5 . I don't propose rebasing z5py on that, but h5py_like's shape_utils module may be of interest: I started from the z5py implementation but have extended it in a couple of ways.

Specifically, it handles non-1 striding, and newaxes. It does it in a naive way: it lets the underlying library read out the unstrided data, and then strides (including reversing) in numpy if necessary (which is presumably a copy operation). Newaxes and squeezes are then applied by reshaping, which I think just returns a view.

clbarnes avatar Jun 26 '19 22:06 clbarnes

Specifically, it handles non-1 striding, and newaxes. It does it in a naive way: it lets the underlying library read out the unstrided data, and then strides (including reversing) in numpy if necessary (which is presumably a copy operation).

Cool! Supporting this in z5py directly would also be nice.

It does it in a naive way: it lets the underlying library read out the unstrided data,

I don't think we can do anything more clever here, so this sounds like a good solution.

A PR for strided views and newaxis support would be very welcome ;).

constantinpape avatar Jun 27 '19 07:06 constantinpape