risingwave icon indicating copy to clipboard operation
risingwave copied to clipboard

feat(storage): separate key and values in imm to reduce allocation

Open wenym1 opened this issue 1 year ago • 0 comments

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Previously, to store values of multiple versions of a key in imm, we use a two-dimensional vector to store the values, where the outer vector stores all entries and each inner vector stores the multi-version values. However, each inner vector is small, and when imm merge is disabled, each inner vector only stores a single value, which involves unnecessary vector allocation. In this PR, we change to store the values of multi-version of all keys in a single vector. Each key entry will store an extra value offset to indicate the starting offset of its own values. The end offset is the value offset of the next entry, or the vector end if the current entry is the last one.

Checklist

  • [ ] I have written necessary rustdoc comments
  • [ ] I have added necessary unit tests and integration tests
  • [ ] I have added test labels as necessary. See details.
  • [ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
  • [ ] My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • [x] All checks passed in ./risedev check (or alias, ./risedev c)
  • [ ] My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • [ ] My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • [ ] My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

wenym1 avatar Feb 27 '24 11:02 wenym1