s4sarath

Hi , The learning rate mentioned in paper for summarization is around 3e-5 . But in the run_summarization.py it is mentioned as 0.32 ( default ) in the flags. In...

DDP (multi GPU) Iterable Dataset is not working as expected ?

40

### Bug description Hi, I am currently testing with IterableDataset and DDP. Total Examples - ```10000``` Batch_size - ```32``` NUM_GPUS - ```2``` . While using IterableDataset , ideally with 2...