cucim
cucim copied to clipboard
[FEA] Support Cloud-native filesystem (S3, GCS)
Is your feature request related to a problem? Please describe.
It would be great if cuCIM's input can be a pointer(URL) of remote storage.
Describe the solution you'd like
For remote/network file system using RDMA, cuFile (GDS) API can be used. For Cloud storage file system, we can leverage existing libraries in C++ or Python (https://fsspec.github.io/kerchunk/, https://github.com/fsspec/filesystem_spec ).
Additional context
- rasterio implements virtual file system in C++ : https://rasterio.readthedocs.io/en/latest/topics/vsi.html
- tifffile supports Zarr and filesystem_spec: https://github.com/cgohlke/tifffile/issues/56
- tiffslide that uses tifffile supports filesystem_spec : https://github.com/bayer-science-for-a-better-life/tiffslide#access-files-in-the-cloud
- https://github.com/rapidsai/cucim/issues/94
- https://github.com/rapidsai/cucim/issues/221