zeus icon indicating copy to clipboard operation
zeus copied to clipboard
verified publisher icon ml-energy

Training framework integration opportunities

Open jaywonchung opened this issue 1 year ago • 1 comments

  • PipelineFrequencyOptimizer: Large model training frameworks

    • Deepspeed
    • Megatron-LM
    • Deepspeed-Megatron
    • GPT-NeoX
    • Reuse training examples in each large model training framework (e.g., Llama pre-training or fine-tuning)

  • GlobalPowerLimitOptimizer

    • PyTorch FSDP
    • Deepspeed ZeRO

jaywonchung avatar May 16 '24 15:05 jaywonchung

PyTorch FSDP integration: #147

jaywonchung avatar Dec 12 '24 17:12 jaywonchung