pipeline-parallelism topic

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

bigscience-workshop

Serving Inside Pytorch

torchpipe

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

xrsrke

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines.

Shigangli

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train...

Coobiw

InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.

InternLM