dlio_benchmark icon indicating copy to clipboard operation
dlio_benchmark copied to clipboard

Published 20 hours ago • verified publisher icon argonne-lcf

Redundant shuffling

Open zhenghh04 opened this issue 5 months ago • 0 comments

In reconfigure, global shuffling is performed. The data loader only have the indices of the local samples. It still has the shuffling on the pytorch / dali data loader which need to be fixed.

Please send another PR to fixed the redundant shuffling issue @hariharan-devarajan

zhenghh04 avatar Aug 27 '24 22:08 zhenghh04