LLMTrainer icon indicating copy to clipboard operation
LLMTrainer copied to clipboard

Published 20 hours ago • verified publisher icon ifromeast
→

Metadata

A comparison of pretraining framework for LLM

LLMTrainer

对比 LLM 的多种预训练方案的性能，方案包括：

  • Torch DDP
  • FSDP
  • DeepSpeed
  • FlashAttention V2
  • ColossalAI
  • Sophia Optimizer
  • Lightning Fabric

详细介绍见 https://zhuanlan.zhihu.com/p/648094197?

← Metadata

19
Stars
3
Forks
Watchers

Owner

verified publisher icon ifromeast

Metadata

A comparison of pretraining framework for LLM

Back