text icon indicating copy to clipboard operation
text copied to clipboard

Verbose argument for dataset loading

Open knagrecha opened this issue 4 years ago • 2 comments

🚀 Feature

Adding a verbose argument for dataset loading to give progress updates while loading a dataset.

Sometimes these datasets can be huge and the user is waiting a LONG time. Having a progress bar of some sort would let the user debug failures due to OOM issues, or even just reassure them that something is happening!

This is a very simple feature, but it would help give users a better understanding of their processes and provide some peace of mind while running large experiments.

knagrecha avatar Feb 10 '21 19:02 knagrecha

@ejguan , @kevinchn : I wonder if we add progress bar to download data pipes?

parmeet avatar Jun 23 '22 22:06 parmeet

We do have a tracking issue here: https://github.com/pytorch/data/issues/457

But, based on my understanding, we might not have the bandwidth for now to implement it.

ejguan avatar Jun 24 '22 13:06 ejguan