kvikio icon indicating copy to clipboard operation
kvikio copied to clipboard

Consider subclassing Python `CuFile` object from `io.RawIOBase`

Open jakirkham opened this issue 1 year ago • 4 comments

To improve interoperability between CuFile objects and other Python libraries that accept file objects, it would be nice to subclass io.RawIOBase and implement the methods io.RawIOBase defines

This would allow consumers of KvikIO to create a CuFile object and leverage it with existing Python libraries implementing custom file readers/writers. It would also allow user to interweave any header/footer parsing of metadata with KvikIO (perhaps on to CPU) and then follow up by reading the bulk data after with KvikIO (likely on to GPU)

More generally this would allow users to compose KvikIO's CuFile object with other Python libraries to quickly handle reading more file formats onto the GPU without needing to reimplement each file format/library

jakirkham avatar Dec 10 '24 23:12 jakirkham

A subset of this functionality (implementing seek and tell specifically) is mentioned in issue: https://github.com/rapidsai/kvikio/issues/576

jakirkham avatar Dec 10 '24 23:12 jakirkham

Hi @jakirkham @madsbk , Do you think we can add some examples/tutorials in https://github.com/rapidsai/kvikio/tree/branch-25.02/notebooks to showcase the applications on CuFile for medical image loading?

According to my current researches and experiments, it's feasible to do:

  1. use kvikio + nibabel to load .nii images faster
  2. use kvikio + pydicom to load .dcm images faster

I'm fine to make some contributions if you agree. Welcome to have further discussions!

yiheng-wang-nv avatar Dec 16 '24 04:12 yiheng-wang-nv

Sure @yiheng-wang-nv, that would be great!

madsbk avatar Dec 19 '24 07:12 madsbk

Thanks @madsbk for the support. I created a ticket: https://github.com/rapidsai/kvikio/issues/580 to track the progress.

yiheng-wang-nv avatar Dec 23 '24 04:12 yiheng-wang-nv