conduit icon indicating copy to clipboard operation
conduit copied to clipboard

Make Relay support 3D HDF5 hyperslabs

Open agcapps opened this issue 1 year ago • 3 comments

Currently, Relay sets the offsets and strides using only the first element of whatever the user passed in. We should support N-D hyperslabs, that is, whatever the user passes in. Additionally, we should support element counts in N-D, not just "number of elements."

agcapps avatar Apr 25 '23 22:04 agcapps

In a little greater detail, I propose to allow the user to specify offset, stride, and size as length-n arrays. The code that prepares to call H5Sselect_hyperslab() would properly use these arrays instead of hardcoding length-1 arrays. The challenge is how to do this without allocation.

agcapps avatar Apr 27 '23 18:04 agcapps

Node opts, n;
int offset[3] = {1, 0, 0};
int count[3] = {2, 2, 4};
opts["offset"].set(offset);
opts["count"].set(count);
int output[16];
n.set(output);

// HDF5 object at path "thefile.hdf5:myobj" has size (4, 3, 4), 
// that is, four slabs with three rows of four elements
// The following should read the middle two slabs, the first two rows, all four elements:

io::hdf5_read("thefile.hdf5:myobj", opts, n);

This assumes that the "shape" of in-memory hyperslab will always be linear. opts always apply to the HDF5 hyperslab: when passed to hdf5_read(), the source; when passed to hdf5_write(), the destination. I can imagine use cases for specifying the in-memory hyperslab beyond linear, but specifying the HDF5 hyperslab is a good start and needed as a feature right now.

agcapps avatar May 15 '23 22:05 agcapps

PR #1129 started out as read/write but I reduced its scope to just reading. I'll add writing in a subsequent PR.

agcapps avatar Jun 28 '23 21:06 agcapps