Haolan

Hi, I'd like to build this chunkeddataset support if no one else is doing it. :-) My own project also needs that support.

I have some questions regarding the original design, why do we need to implement the abstract method `process_chunk`? Can't we just take a similar approach like `InMemoryDataset`, asking users to...

So basically my current thoughts are that users need to write the chunk logic in `process()` and store them in some data structures such as `self.chunked_data` and `self.chunked_slices` (both are...

> Is anyone working on this? > > For my own purposes, I'm currently implementing logic to save processed data as chunks to disk and load on-the-fly to a dataset...

Has a similar problem.