【AutoParallel】Add `sharding_comm_overlap`

[heavyrain-lzy]

PR Category

Distributed Strategy

PR Types

Performance

Description

Pcard-76459 Add sharding_comm_overlap and change the way of gradient_syn_after_accumulate.

[paddle-bot[bot]]

你的PR提交成功，感谢你对开源项目的贡献! 请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。 Your PR has been submitted. Thanks for your contribution! Please wait for the result of CI firstly. See Paddle CI Manual for details.

[paddle-ci-bot[bot]]

Sorry to inform you that ada1d9e's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}