Shigang Li
Results
2
repositories owned by
Shigang Li
Chimera
37
Stars
6
Forks
Watchers
Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines.
Ok-Topk
20
Stars
8
Forks
Watchers
Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k communication volume which is asymptotically optimal) with the d...