risingwave icon indicating copy to clipboard operation
risingwave copied to clipboard

refactor(streaming): stream agg only read and write value column

Open wcy-fdu opened this issue 3 years ago • 2 comments
trafficstars

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

As we have changed streaming table's read and write interfaces to handle value indices (default is full columns), this PR changes stream agg to only read and write one value column. Previous PR have refactor hash_join's degree table.

Checklist

  • [x] I have written necessary rustdoc comments
  • [x] I have added necessary unit tests and integration tests
  • [x] All checks passed in ./risedev check (or alias, ./risedev c)

Documentation

If your pull request contains user-facing changes, please specify the types of the changes, and create a release note. Otherwise, please feel free to remove this section.

Types of user-facing changes

Please keep the types that apply to your changes, and remove those that do not apply.

  • Installation and deployment
  • Connector (sources & sinks)
  • SQL commands, functions, and operators
  • RisingWave cluster configuration changes
  • Other (please specify in the release note below)

Release note

Please create a release note for your changes. In the release note, focus on the impact on users, and mention the environment or conditions where the impact may occur.

Refer to a related PR or issue link (optional)

https://github.com/risingwavelabs/risingwave/issues/5101

wcy-fdu avatar Sep 18 '22 08:09 wcy-fdu

Codecov Report

Merging #5411 (686a915) into main (cb0d309) will increase coverage by 0.01%. The diff coverage is 87.83%.

@@            Coverage Diff             @@
##             main    #5411      +/-   ##
==========================================
+ Coverage   74.29%   74.30%   +0.01%     
==========================================
  Files         924      923       -1     
  Lines      144263   144182      -81     
==========================================
- Hits       107174   107130      -44     
+ Misses      37089    37052      -37     
Flag Coverage Δ
rust 74.30% <87.83%> (+0.01%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/storage/src/error.rs 0.00% <ø> (ø)
src/storage/src/table/mod.rs 59.52% <ø> (ø)
src/stream/src/executor/error.rs 5.88% <ø> (+0.24%) :arrow_up:
...c/storage/src/table/streaming_table/state_table.rs 82.56% <81.48%> (-0.11%) :arrow_down:
.../storage/src/hummock/sstable/sstable_id_manager.rs 89.28% <87.50%> (-0.14%) :arrow_down:
...ge/src/hummock/local_version/local_version_impl.rs 79.65% <88.46%> (-0.66%) :arrow_down:
...rc/frontend/src/optimizer/plan_node/logical_agg.rs 93.89% <100.00%> (+<0.01%) :arrow_up:
...rage/src/hummock/local_version/flush_controller.rs 60.82% <100.00%> (-0.14%) :arrow_down:
...src/hummock/local_version/local_version_manager.rs 77.16% <100.00%> (-0.06%) :arrow_down:
src/stream/src/executor/aggregation/mod.rs 85.71% <100.00%> (-0.17%) :arrow_down:
... and 2 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov[bot] avatar Sep 18 '22 08:09 codecov[bot]

this PR changes the definition of column_mapping. I think we need more discussion later.

st1page avatar Sep 19 '22 06:09 st1page

e2e tests do not seem to cover this change, related to https://github.com/risingwavelabs/risingwave/issues/5080

wcy-fdu avatar Oct 09 '22 04:10 wcy-fdu

e2e tests do not seem to cover this change, related to https://github.com/risingwavelabs/risingwave/issues/5080

The newly introduced scaling test will cover this. :)

BugenZhao avatar Oct 09 '22 04:10 BugenZhao

MaterializedInputStates seem to need this too, I may do that later, together with some other refactoring.

stdrc avatar Oct 09 '22 04:10 stdrc