liuyoude issues

Repositories
Issues
Comments

Results 1 issues of


                                            liuyoude

How to gather cluster_size and embed_sum when use DataParallel (DP)

I notice the code can gather data for EMA in DDP, but I filed it when I use DP, because distributed.all_reduce need distributed.init_process_group firstly. How to gather data in DP