refactor: replace Bytes with Vec<u8> for object store to avoid buffer copy when decoding block without compression

[MrCroxx]

trafficstars

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

After #4484 , we use Vec<u8> as underlying buffer storage for Block to control memory usage. But with Bytes returned by object store, there is always a buffer copy when decoding even if there is no compression.

This PR refactored object store interface and replace Bytes with Vec<u8>.

Pros:

• Avoid buffer copy when decoding Block if there's no compression.
• No extra cost on object store side.

Cons:

• There is an extra buffer copy in Sstable::new_with_data(), which is unavoidable.

TODO:

• [ ] Benchmarks to compare the cost.

Checklist

• [x] I have written necessary rustdoc comments
• [x] I have added necessary unit tests and integration tests
• [x] All checks passed in ./risedev check (or alias, ./risedev c)

Refer to a related PR or issue link (optional)

#4484 #4259

[codecov[bot]]

Codecov Report

Merging #4565 (1dc39e8) into main (5796692) will decrease coverage by 0.03%. The diff coverage is 69.31%.

@@            Coverage Diff             @@
##             main    #4565      +/-   ##
==========================================
- Coverage   74.23%   74.19%   -0.04%     
==========================================
  Files         854      854              
  Lines      127235   127316      +81     
==========================================
+ Hits        94448    94461      +13     
- Misses      32787    32855      +68     

Flag	Coverage Δ
rust	74.19% <69.31%> (-0.04%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/connector/src/source/kafka/mod.rs	0.00% <ø> (ø)
src/connector/src/source/kinesis/mod.rs	0.00% <ø> (ø)
src/connector/src/source/pulsar/mod.rs	0.00% <ø> (ø)
src/ctl/src/cmd_impl/hummock/sst_dump.rs	0.00% <0.00%> (ø)
src/expr/src/expr/expr_unary.rs	68.86% <ø> (ø)
src/expr/src/vector_op/cast.rs	72.92% <0.00%> (-2.08%)	:arrow_down:
src/frontend/src/handler/util.rs	80.28% <ø> (-1.54%)	:arrow_down:
...frontend/src/optimizer/plan_node/batch_exchange.rs	63.82% <ø> (-0.76%)	:arrow_down:
src/meta/src/manager/catalog.rs	17.22% <0.00%> (ø)
src/meta/src/manager/notification.rs	55.90% <0.00%> (ø)
... and 56 more

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

[hzxa21]

Sstable::new_with_data will be removed in #4590. We can merge #4590 first and refactor this PR accordingly.

[fuyufjh]

Hi, any updates?

[MrCroxx]

Close, merge #4982 instead.