pipeline-parallelism topic

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

microsoft

Making large AI models cheaper, faster and more accessible

hpcaitech

飞桨大模型开发套件，提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。

PaddlePaddle

LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training

Oneflow-Inc

Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.

alibaba

A GPipe implementation in PyTorch

kakaobrain

An Efficient Pipelined Data Parallel Approach for Training Large Model

AlibabaPAI

Implementation of autoregressive language model using improved Transformer and DeepSpeed pipeline parallelism.

nawnoes

FTPipe and related pipeline model parallelism research.

saareliad

A curated list of awesome projects and papers for distributed training or inference

Shenggan