returnn icon indicating copy to clipboard operation
returnn copied to clipboard

PyTorch DataLoader, use num_workers=1 by default?

Open albertz opened this issue 2 years ago • 0 comments

Pro num_workers=1:

  • It should always be faster, usually fix low computation time (#1383, #1435).
  • It should just work.

Contra, i.e. leave the current default, num_workers=0:

  • It is different from the PyTorch default.
  • There might be rare cases where it does not work? (But I don't know any. I think all RETURNN datasets should work just fine.)
  • Debugging might be more difficult.
  • It's more complex logic.
  • It adds some memory overhead.

albertz avatar Oct 17 '23 16:10 albertz