MLDatasets.jl
MLDatasets.jl copied to clipboard
Add `transform` keyword to dataset constructors to store a transformation
Both HuggingFace and torchvision's datasets can store a transformation that is applied on the fly when indexing a dataset. I think this is generally very convenient and we should add it to every dataset.
I feel it may be cleaner to tell users to use mapobs
and give some examples. AFAICT there would be no functional difference, right?
Yes maybe there is no advantage over a mapobs
. I'll keep the issue open until we add some examples in the docs.
The recently introduced TorchData could be a source of inspiration https://pytorch.org/data/beta/index.html