How does shuffle works in sampler

[yinruiqing]

❓ Questions & Help

I think shuffle method cannot be called by pytorch lightning. That may also cause some problems with 'ddp' mode.

[sooftware]

Can you explain in more detail why?

[yinruiqing]

You can find how pl set distributed sampler here. If you dive into the source code of DistributedSampler in pytorch, you can find the random seed depends on the epoch. Pytorch recommends to call set_epoch() at start of each epoch. But I don't know how to do it in pl. If you want shuffle() work, you should also call it explictly at the start of each epoch. In addition, you should also set replace_sampler_ddp=True in trainer

[sooftware]

I'll check. thanks.