risingwave icon indicating copy to clipboard operation
risingwave copied to clipboard

feat(storage): introduce filter_key_extractor for sst builder

Open Li0k opened this issue 3 years ago • 3 comments

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

as title

Please explain IN DETAIL what the changes are in this PR and why they are needed:

  • support filter_key_extractor for sst_builder

Checklist

  • ~[x] I have written necessary rustdoc comments~
  • ~[x] I have added necessary unit tests and integration tests~
  • ~[x] All checks passed in ./risedev check (or alias, ./risedev c)~

Refer to a related PR or issue link (optional)

  • https://github.com/singularity-data/risingwave/issues/3871

Li0k avatar Aug 03 '22 06:08 Li0k

Codecov Report

Merging #4394 (eaef6ba) into main (91c22d7) will increase coverage by 0.08%. The diff coverage is 89.10%.

@@            Coverage Diff             @@
##             main    #4394      +/-   ##
==========================================
+ Coverage   74.15%   74.23%   +0.08%     
==========================================
  Files         861      861              
  Lines      128178   128506     +328     
==========================================
+ Hits        95052    95401     +349     
+ Misses      33126    33105      -21     
Flag Coverage Δ
rust 74.23% <89.10%> (+0.08%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/batch/src/task/task_execution.rs 60.07% <ø> (-0.31%) :arrow_down:
src/frontend/src/catalog/table_catalog.rs 98.12% <ø> (-0.04%) :arrow_down:
...tend/src/optimizer/plan_node/stream_materialize.rs 90.60% <ø> (-0.06%) :arrow_down:
src/frontend/src/optimizer/plan_node/utils.rs 83.63% <ø> (-0.15%) :arrow_down:
src/storage/src/lib.rs 100.00% <ø> (ø)
src/storage/src/monitor/monitored_store.rs 1.58% <0.00%> (-0.04%) :arrow_down:
src/storage/src/panic_store.rs 0.00% <0.00%> (ø)
src/storage/src/store.rs 57.14% <ø> (ø)
...rc/storage/hummock_sdk/src/filter_key_extractor.rs 89.76% <48.38%> (-1.58%) :arrow_down:
src/storage/src/hummock/state_store.rs 80.71% <58.82%> (+4.53%) :arrow_up:
... and 42 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov[bot] avatar Aug 10 '22 10:08 codecov[bot]

bench_compactor

  • main image

  • branch image

Li0k avatar Aug 11 '22 10:08 Li0k

replace to_vec with extend_from_slice for last_full_key

image

Li0k avatar Aug 11 '22 11:08 Li0k

at tpch-q12 (branch on the left, main on the right), we can observe some improvements

image

image

image

image

Li0k avatar Aug 15 '22 06:08 Li0k