sequence-parallelism topic

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

xrsrke

InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.

InternLM