text
text copied to clipboard
Verbose argument for dataset loading
🚀 Feature
Adding a verbose argument for dataset loading to give progress updates while loading a dataset.
Sometimes these datasets can be huge and the user is waiting a LONG time. Having a progress bar of some sort would let the user debug failures due to OOM issues, or even just reassure them that something is happening!
This is a very simple feature, but it would help give users a better understanding of their processes and provide some peace of mind while running large experiments.
@ejguan , @kevinchn : I wonder if we add progress bar to download data pipes?
We do have a tracking issue here: https://github.com/pytorch/data/issues/457
But, based on my understanding, we might not have the bandwidth for now to implement it.