incubator-uniffle icon indicating copy to clipboard operation
incubator-uniffle copied to clipboard

[#2697] refactor(spark): Involve related writer stats info into ShuffleWriteTaskStats

Open zuston opened this issue 3 weeks ago • 1 comments

What changes were proposed in this pull request?

Involve related writer stats info into ShuffleWriteTaskStats for further integrity validation block checksum implementation

Why are the changes needed?

for #2697

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing unit tests

zuston avatar Dec 10 '25 07:12 zuston

Test Results

 3 175 files  +10   3 175 suites  +10   6h 58m 58s ⏱️ + 2m 10s  1 237 tests + 5   1 236 ✅ + 5   1 💤 ±0  0 ❌ ±0  15 674 runs  +55  15 659 ✅ +55  15 💤 ±0  0 ❌ ±0 

Results for commit 691c7ee8. ± Comparison against base commit c7e23b6b.

This pull request removes 4 and adds 9 tests. Note that renamed tests count towards both.
org.apache.uniffle.storage.handler.impl.PrefetchableClientReadHandlerTest ‑ test_with_fetch_failure
org.apache.uniffle.storage.handler.impl.PrefetchableClientReadHandlerTest ‑ test_with_prefetch
org.apache.uniffle.storage.handler.impl.PrefetchableClientReadHandlerTest ‑ test_with_timeout
org.apache.uniffle.storage.handler.impl.PrefetchableClientReadHandlerTest ‑ test_without_prefetch
org.apache.uniffle.shuffle.BlockStatsTest ‑ testConstructor
org.apache.uniffle.shuffle.BlockStatsTest ‑ testMerge
org.apache.uniffle.shuffle.BlockStatsTest ‑ testRemove
org.apache.uniffle.shuffle.BlockStatsTest ‑ testRemoveMultipleBlocks
org.apache.uniffle.storage.handler.impl.PrefetchableClientReadHandlerTest ‑ testWithPrefetch
org.apache.uniffle.storage.handler.impl.PrefetchableClientReadHandlerTest ‑ testWithPrefetchButFailure
org.apache.uniffle.storage.handler.impl.PrefetchableClientReadHandlerTest ‑ testWithPrefetchButTimeout
org.apache.uniffle.storage.handler.impl.PrefetchableClientReadHandlerTest ‑ testWithPrefetchWithEmptyResult
org.apache.uniffle.storage.handler.impl.PrefetchableClientReadHandlerTest ‑ testWithoutPrefetch

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Dec 10 '25 08:12 github-actions[bot]

@zuston Some copilot comments are useful. You can take a look.

jerqi avatar Dec 15 '25 12:12 jerqi

@zuston Some copilot comments are useful. You can take a look.

yes. all done

zuston avatar Dec 16 '25 08:12 zuston