incubator-uniffle icon indicating copy to clipboard operation
incubator-uniffle copied to clipboard

[#1658] fix(server): FileSegmentManagedBuffer#nioByteBuffer read result cacheable

Open xumanbu opened this issue 10 months ago • 6 comments

What changes were proposed in this pull request?

FileSegmentManagedBuffer#nioByteBuffer add a buffer to hold read file result to avoid multiple read file

Why are the changes needed?

Fix: #1658

Does this PR introduce any user-facing change?

No.

How was this patch tested?

existed UT

xumanbu avatar Apr 18 '24 07:04 xumanbu

PTAL. @dingshun3016 @zuston @jerqi

xumanbu avatar Apr 18 '24 08:04 xumanbu

Test Results

 2 398 files  +14   2 398 suites  +14   4h 38m 33s :stopwatch: + 6m 5s    921 tests + 2     920 :white_check_mark: + 2   1 :zzz: ±0  0 :x: ±0  10 689 runs  +23  10 675 :white_check_mark: +23  14 :zzz: ±0  0 :x: ±0 

Results for commit 8740feb4. ± Comparison against base commit 5ab625b5.

This pull request removes 1 and adds 3 tests. Note that renamed tests count towards both.
org.apache.uniffle.shuffle.manager.ShuffleManagerServerFactoryTest ‑ testShuffleManagerServerType
org.apache.uniffle.common.netty.buffer.FileSegmentManagedBufferTest ‑ testNioByteBuffer{File}
org.apache.uniffle.shuffle.manager.ShuffleManagerServerFactoryTest ‑ testShuffleManagerServerType{ServerType}[1]
org.apache.uniffle.shuffle.manager.ShuffleManagerServerFactoryTest ‑ testShuffleManagerServerType{ServerType}[2]

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Apr 18 '24 08:04 github-actions[bot]

Could you help review this? @rickyma

zuston avatar Apr 18 '24 08:04 zuston

Left some comments. I'm OK with this patch. But you need to test it more carefully. Because this is used in gRPC mode. It won't be invoked in Netty mode. So I don't know how it actually performs in a production environment. Because I currently use Netty in my environment.

Oh I thought this is netty. I will take another look tomorrow.

zuston avatar Apr 18 '24 13:04 zuston

Could you add a ut if this is a fix?

jerqi avatar Apr 19 '24 12:04 jerqi

Could you add a ut if this is a fix?

ok. added.

xumanbu avatar Apr 29 '24 06:04 xumanbu