risingwave icon indicating copy to clipboard operation
risingwave copied to clipboard

feat(storage): remove FilterKeyExtractor trait for MultiFilterKeyExtractor

Open Li0k opened this issue 3 years ago • 2 comments

Is your feature request related to a problem? Please describe. Currently, MultiFilterKeyExtractor is a struct that impletement FilterKeyExtractor trait. We can provide a nested structure with this implementation, but this is not safe. We want FilterKeyExtractor to be stateless, but now MultiFilterKeyExtractor needs to hold mutable self in order to maintain state. (For borrow checker, we use need a Mutex even if it wont be competing).

(https://github.com/singularity-data/risingwave/blob/main/src/storage/hummock_sdk/src/filter_key_extractor.rs) simple conclusion:

  1. FilterKeyExtractor to be stateless
  2. FilterKeyExtractor no needs to hold mutable self for maintaining state

Describe the solution you'd like

  1. use a new struct FilterKeyExtractorContext to replace MultiFilterKeyExtractor which maintain state.
  2. remove MultiFilterKeyExtractor for avoiding nested structure that it's uncontrollable
  3. cached the last_filter_key_extractor_state for query optimization.
  4. use separate FilterKeyExtractorContext for every compaction task split (Copy)

Li0k avatar Aug 19 '22 07:08 Li0k

May I take a look? I may be able to submit a PR next week...👀 Won't be fast...

ClSlaid avatar Sep 21 '22 09:09 ClSlaid

May I take a look? I may be able to submit a PR next week...👀 Won't be fast...

Thanks, you can self-assign

Li0k avatar Sep 21 '22 10:09 Li0k

  1. FilterKeyExtractor to be stateless
  2. FilterKeyExtractor no needs to hold mutable self for maintaining state

wait...? a stateless structure but need to maintain state?🤯

ClSlaid avatar Oct 02 '22 03:10 ClSlaid