PaddleNLP
PaddleNLP copied to clipboard
[Performance] Optimize unified checkpoint save/load speed.
PR types
PR changes
Description
For facebook/llama-7b
UC/PD Save/Load 对比 | UC save | PD save |
---|---|---|
testDP8 | 0 | 0 |
testPP4DP2 | 47.13 | 68.36 |
testPP4Sharding2 | 25.93 | 41.04 |
testPP8 | 27.04 | 39.68 |
testSharding2S1DP4 | 93.19 | 145.13 |
testSharding2S2DP4 | 90.62 | 147.15 |
testSharding4S1DP2 | 47.6 | 94.75 |
testSharding4S2DP2 | 44.12 | 95.68 |
testSharding8S1 | 25.94 | 64.37 |
testSharding8S2 | 23.95 | 65.77 |
testTP2PP4 | 27.92 | 35.71 |
testTP2Sharding4 | 27.78 | 46.31 |
testTP4DP2 | 53.74 | 51.98 |
testTP4PP2 | 27.32 | 30.55 |
testTP4Sharding2 | 28.35 | 31.31 |
testTP8 | 30.63 | 23.75 |
Thanks for your contribution!
Codecov Report
Attention: Patch coverage is 73.49398%
with 66 lines
in your changes are missing coverage. Please review.
Project coverage is 55.42%. Comparing base (
f29a7b9
) to head (32132ae
). Report is 23 commits behind head on develop.
Additional details and impacted files
@@ Coverage Diff @@
## develop #8204 +/- ##
===========================================
+ Coverage 55.37% 55.42% +0.04%
===========================================
Files 613 615 +2
Lines 95855 96235 +380
===========================================
+ Hits 53083 53335 +252
- Misses 42772 42900 +128
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.