ring-attention topic

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

feifeibear

InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.

InternLM

[CVPR 2025 Highlight] The official CLIP training codebase of Inf-CL: "Breaking the Memory Barrier: Near Infinite Batch Size Scaling for Contrastive Loss". A super memory-efficiency CLIP training schem...

DAMO-NLP-SG