Fabrizio Ottati

Consider #167, and given the possibility provided by DataPipes, I was thinking about giving the user the possibility to apply externally the transform to the events, by making they aware...

I agree with you on waiting for what they with their transforms and then copy them :) I see that they are creating [dataset classes from `IterDataPipe`](https://github.com/pytorch/vision/blob/main/torchvision/prototype/datasets/utils/_dataset.py), hence they do...

I don't know about you @biphasic but I completely missed [this torchvision folder](https://github.com/pytorch/vision/tree/main/torchvision/prototype/datasets/_builtin) in which they have already implemented with data pipes many datasets :laughing:

I'll try to summarize here what I understood studying the torchvision code. Their [README](https://github.com/pytorch/vision/blob/main/torchvision/prototype/datasets/_builtin/README.md) carefully explains how one should implement a datapiped dataset, hence I used it as inspiration for...

@biphasic should I try to add some testing routines for the prototypes?

OK, I will try to come up with something. In the meanwhile, I tried to use their web datapipes and they seem pretty nice. Basically, you set up a datapipe...

I would have left it in the `prototype/utils` folder as temporary replacement, given also the fact that torchivision is switching to SHA256 check from MD5 (in fact, a `check_sha256` function...

Well, actually there seems to be no slowdown if the decoding function is included in the pipe... If you remove the encoding step, the ratio zipped/unzipped load times is ~6....

I just have one doubt. I benchmarked over 1000 samples the read speed.  Right now, we are using the archive loaders from torchdata to iterate through the dataset. With...

I think you are absolutely right. Then, let's decompress the data by default. I want to understand if this can be accomplished trough torchdata utilities, so that we have to...