incubator-uniffle icon indicating copy to clipboard operation
incubator-uniffle copied to clipboard

fix(spark): add all faulty servers into blacklist while writing for stage recomputing

Open zuston opened this issue 1 year ago • 6 comments

What changes were proposed in this pull request?

add all faulty servers into blacklist while writing for stage recomputing

Why are the changes needed?

It will fail on retry when only one server is added into blacklist.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing tests.

zuston avatar Mar 14 '24 09:03 zuston

Could you help review this? @yl09099

zuston avatar Mar 14 '24 09:03 zuston

Codecov Report

Attention: Patch coverage is 50.00000% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 54.95%. Comparing base (f6569b0) to head (1f1d00c). Report is 5 commits behind head on master.

Files Patch % Lines
...fle/shuffle/manager/ShuffleManagerGrpcService.java 50.00% 1 Missing and 1 partial :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #1578      +/-   ##
============================================
+ Coverage     53.99%   54.95%   +0.96%     
  Complexity     2862     2862              
============================================
  Files           438      418      -20     
  Lines         24819    22480    -2339     
  Branches       2111     2113       +2     
============================================
- Hits          13402    12355    -1047     
+ Misses        10578     9355    -1223     
+ Partials        839      770      -69     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Mar 14 '24 09:03 codecov-commenter

Test Results

 2 315 files   - 25   2 315 suites   - 25   3h 57m 13s :stopwatch: - 34m 43s    892 tests  - 16     891 :white_check_mark:  - 16   1 :zzz: ±0  0 :x: ±0  10 508 runs   - 33  10 494 :white_check_mark:  - 33  14 :zzz: ±0  0 :x: ±0 

Results for commit 1f1d00c8. ± Comparison against base commit f6569b04.

This pull request removes 16 tests.
org.apache.uniffle.test.DynamicConfTest ‑ dynamicConfTest
org.apache.uniffle.test.HadoopConfTest ‑ hadoopConfTest
org.apache.uniffle.test.LargeSorterTest ‑ largeSorterTest
org.apache.uniffle.test.SecondarySortTest ‑ secondarySortTest
org.apache.uniffle.test.TezCartesianProductTest ‑ cartesianProductTest
org.apache.uniffle.test.TezHashJoinTest ‑ hashJoinDoBroadcastTest
org.apache.uniffle.test.TezHashJoinTest ‑ hashJoinTest
org.apache.uniffle.test.TezOrderedWordCountTest ‑ orderedWordCountTest
org.apache.uniffle.test.TezSimpleSessionExampleTest ‑ simpleSessionExampleTest
org.apache.uniffle.test.TezSortMergeJoinTest ‑ sortMergeJoinTest
…

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Mar 14 '24 09:03 github-actions[bot]

LGTM~

yl09099 avatar Mar 17 '24 15:03 yl09099

Do you add a new ut for this case?

jerqi avatar Mar 18 '24 01:03 jerqi

Do you add a new ut for this case?

I'm still reviewing this part.

zuston avatar Mar 18 '24 06:03 zuston