PaddleMIX
PaddleMIX copied to clipboard
support auto parallel in dit and largedit
trafficstars
support auto parallel in model of Dit and LargeDit, include:
- support model of Dit and LargeDit to be trained using auto parallel in dynamic graph mode (hereinafter referred to as
auto). - support model of Dit and LargeDit to be trained using auto parallel in dynamic to static graph mode with pir (hereinafter referred to as
auto-static). - support hybrid parallel(DP、Sharding、MP) in model of Dit and LargeDit, PP under verification.
compare loss of manual parallel(present, hereinafter referred to as manual), auto and auto-static
compare loss diff of manual and auto
compare loss diff of manual and auto-static
compare loss diff of auto and auto-static
Thanks for your contribution!
https://github.com/PaddlePaddle/PaddleMIX/tree/develop/ppdiffusers/examples/class_conditional_image_generation/DiT 在这里新增自动并行的相关文档吧
感谢提醒,已增加使用自动并行进行训练的文档说明