PaddleMIX icon indicating copy to clipboard operation
PaddleMIX copied to clipboard

support auto parallel in dit and largedit

Open jeff41404 opened this issue 1 year ago • 3 comments
trafficstars

support auto parallel in model of Dit and LargeDit, include:

  • support model of Dit and LargeDit to be trained using auto parallel in dynamic graph mode (hereinafter referred to as auto).
  • support model of Dit and LargeDit to be trained using auto parallel in dynamic to static graph mode with pir (hereinafter referred to as auto-static).
  • support hybrid parallel(DP、Sharding、MP) in model of Dit and LargeDit, PP under verification.

compare loss of manual parallel(present, hereinafter referred to as manual), auto and auto-static loss_h_s_pir

compare loss diff of manual and auto hand_auto_diff_pir

compare loss diff of manual and auto-static hand_auto_static_diff

compare loss diff of auto and auto-static auto_auto_static_diff

jeff41404 avatar May 23 '24 10:05 jeff41404

Thanks for your contribution!

paddle-bot[bot] avatar May 23 '24 10:05 paddle-bot[bot]

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Sep 19 '24 03:09 CLAassistant

https://github.com/PaddlePaddle/PaddleMIX/tree/develop/ppdiffusers/examples/class_conditional_image_generation/DiT 在这里新增自动并行的相关文档吧

感谢提醒,已增加使用自动并行进行训练的文档说明

jeff41404 avatar Oct 08 '24 10:10 jeff41404