deepspeed-ulysses topic

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

feifeibear

InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.

InternLM