Loading/Saving Directly from GPU

[CSSFrancis]

Describe the functionality you would like to see.

I'm not sure what would be required to do to get this to work with h5py but zarr and Nvidia have a nice little library kvikio which can be used to load files directly from zarr into a GPU. The major benefit of this is that you can reduce the overhead of loading data using a CPU and then transferring to a GPU.

Describe the context

I see this being used in two places:

• Writing directly to zarr object from a GPU which might be much more efficient
• Fast pipelines for data processing.

Additional information

In general I think that GPU processing will only continue to improve a start to outperform CPU processing. It might be good to get an idea of how difficult it is to do something like this.

https://xarray.dev/blog/xarray-kvikio