Longchentong

dist_util.py ---------------------------- old ---------------------------- def sync_params(params): """ Synchronize a sequence of Tensors across ranks from rank 0. """ for p in params: with th.no_grad(): dist.broadcast(p, 0) ------------------------------- new ------------------------------- def...