incubator-uniffle icon indicating copy to clipboard operation
incubator-uniffle copied to clipboard

[#1608][part-6] improvement(spark): verify the sent blocks count

Open zuston opened this issue 1 year ago • 3 comments

What changes were proposed in this pull request?

Verify the sent blocks count in write tasks for spark

Why are the changes needed?

For #1608. After introducing the reassign menchanism, the blocks' stored location will be dynamiclly changed. To ensure possible or potenial bugs, it's necessary to introduce the block count.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing tests are enough to ensure safe

zuston avatar May 11 '24 02:05 zuston

Test Results

 2 391 files  ±0   2 391 suites  ±0   4h 56m 44s :stopwatch: +20s    928 tests ±0     927 :white_check_mark: ±0   1 :zzz: ±0  0 :x: ±0  10 754 runs  ±0  10 740 :white_check_mark: ±0  14 :zzz: ±0  0 :x: ±0 

Results for commit bf09407a. ± Comparison against base commit 313d4e02.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar May 11 '24 03:05 github-actions[bot]

PTAL @jerqi

zuston avatar May 11 '24 06:05 zuston

PTAL @jerqi

Gentle ping @jerqi

zuston avatar May 14 '24 06:05 zuston