kvrocks icon indicating copy to clipboard operation
kvrocks copied to clipboard

TDigest: use rocksdb iterator for centroids iteration.

Open LindaSummer opened this issue 10 months ago • 1 comments

Search before asking

  • [x] I had searched in the issues and found no similar issues.

Motivation

Related Issue

#2558

In TDigest implementation #2741 , we use DummyCentroids to achieve a simple iteration and it should be refactored to rocksdb native iterator as designed.

https://github.com/apache/kvrocks/pull/2741#discussion_r1950452348

Solution

  • Implement a RocksdbCentroids and replace the DummyCentroids.
  • Add more unit tests for encoding, decoding and algorithm.

Are you willing to submit a PR?

  • [x] I'm willing to submit a PR!

LindaSummer avatar Feb 13 '25 14:02 LindaSummer

Hi Team,

I'm working on this issue now. 😊

I will create a new centroid iterator with rocksdb iterator directly.

I will add more unit tests and create some benchmark for quantile.

Best Regards, Edward

LindaSummer avatar Feb 26 '25 13:02 LindaSummer