incubator-uniffle [Bug] msg: Zstd only supports the same type of bytebuffer decompression

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Search before asking

[X] I have searched in the issues and found no similar issues.

Describe the bug

Affects Version(s)

master

Uniffle Server Log Output

No response

Uniffle Engine Log Output

No response

Uniffle Server Configurations

No response

Uniffle Engine Configurations

No response

Additional context

No response

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Dec 07 '23 10:12 zuston

Do you know the managedBuffer is not direct buffer when getting memory by netty? @leixm

Dec 08 '23 03:12 zuston

Do you know the managedBuffer is not direct buffer when getting memory by netty? @leixm

Does this happen when netty is turned on or off? RssShuffleDataIterator line164 will keep uncompressedData the same type as rawData. Why does the above exception still appear?

Dec 08 '23 03:12 leixm

Do you know the managedBuffer is not direct buffer when getting memory by netty? @leixm

Does this happen when netty is turned on or off? RssShuffleDataIterator line164 will keep uncompressedData the same type as rawData. Why does the above exception still appear?

Netty is on.

Dec 08 '23 03:12 zuston

Is this issue related to spark.rss.client.off.heap.memory.enable?

Dec 08 '23 03:12 jerqi

I found sometimes the bytebuffer is not direct after coverting from netty direct bytebuf using bytebuf.nioBuffer()

Dec 08 '23 06:12 zuston

@leixm

Dec 08 '23 06:12 zuston

@leixm

@LuciferYang Do you know more about this point?

Dec 18 '23 01:12 jerqi