data icon indicating copy to clipboard operation
data copied to clipboard

`DataLoader2.shutdown` should guarantee `finalize_iteration` is called for `ReadingService`

Open ejguan opened this issue 2 years ago • 1 comments

🐛 Describe the bug

Per title: https://github.com/pytorch/data/blob/c42587a828d05f24f6f0586d17d3e9d55e1433ed/torchdata/dataloader2/dataloader2.py#L176-L183

Even though the ReadingService should guarantee the finalize_iteration called in finalize, it's better to guard this behavior in DataLoader2 as well.

Versions

main branch

ejguan avatar Oct 11 '22 20:10 ejguan

In addition to it, DLv2 should guarantee that finalize of ReadingService is called, even if Exception raised during iterations.

VitalyFedyunin avatar Oct 11 '22 22:10 VitalyFedyunin