Adapt fully_shard

[Xing-lil]

PR Category

User Experience

PR Types

Others

Description

• Add support for the fully_shard API in PaddleFleet
• Support both manual_parallel and auto_parallel.
• The fully_shard API no longer accepts the optimizer.
• Check on GLM (manual_parallel) and llama3 (auto_parallel) using fully_shard API in PaddleFormers, check same loss, memory, performance compared to the original sharding stage3.

card-92763

[paddle-bot[bot]]

你的PR提交成功，感谢你对开源项目的贡献! 请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。 Your PR has been submitted. Thanks for your contribution! Please wait for the result of CI firstly. See Paddle CI Manual for details.

[codecov-commenter]

Codecov Report

:x: Patch coverage is 89.97429% with 39 lines in your changes missing coverage. Please review. :warning: Please upload report for BASE (develop@5e27f87). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...eta_parallel/sharding/group_sharded_fully_shard.py	89.70%	35 Missing :warning:
...on/paddle/distributed/auto_parallel/fully_shard.py	88.88%	4 Missing :warning:

:x: Your patch status has failed because the patch coverage (89.97%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             develop   #76770   +/-   ##
==========================================
  Coverage           ?   89.97%           
==========================================
  Files              ?        4           
  Lines              ?      389           
  Branches           ?        0           
==========================================
  Hits               ?      350           
  Misses             ?       39           
  Partials           ?        0           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.