data icon indicating copy to clipboard operation
data copied to clipboard

`DataLoader2` does not properly reset iterator when using `PrototypeMultiprocessingReadingService` and prefetch

Open NivekT opened this issue 2 years ago • 0 comments

🐛 Describe the bug

Vision team reported this behavior on Linux

dp = ...  # something with prefetch
dl = DataLoader2(reading_service=PrototypeMultiProcessingReadingService())
for _ in dl:
    # this works fine

for _ in dl:
    # dl is empty now, this never gets hit!

Versions

https://github.com/pytorch/data/pull/815

cc: @VitalyFedyunin

NivekT avatar Oct 12 '22 16:10 NivekT