model-parallelism topic

List model-parallelism repositories

torchshard

293
Stars
15
Forks
Watchers

Slicing a PyTorch Tensor Into Parallel Shards

verified publisher icon kaiyuyue

dnn-distributed

43
Stars
12
Forks
Watchers

Distributed training of DNNs • C++/MPI Proxies (GPT-2, GPT-3, CosmoFlow, DLRM)

verified publisher icon vdutts7

pytorch-transformer-distributed

24
Stars
10
Forks
Watchers

Distributed training (multi-node) of a Transformer model

verified publisher icon hkproj

sc23-dl-tutorial

25
Stars
5
Forks
Watchers

SC23 Deep Learning at Scale Tutorial Material

verified publisher icon NERSC