kvrocks icon indicating copy to clipboard operation
kvrocks copied to clipboard

Refactor TDigest metadata preparation and merging logic into a helper function

Open SharonIV0x86 opened this issue 7 months ago • 3 comments

Search before asking

  • [x] I had searched in the issues and found no similar issues.

Motivation

Currently, in the functions below,

we repeat the same block of code to:

  • Append namespace prefix to the key
  • Acquire a lock
  • Fetch TDigest metadata
  • Merge unmerged nodes (if any)
  • Update metadata and write it back
  • Refresh the snapshot

As new TDigest features are added, the same preprocessing steps (metadata preparation and merging) will likely be needed repeatedly. And this will most likely cause code duplication of around 20-30 lines across multiple functions.

If this change is accepted then the dependent PR (#2887) is based on this refactor, and cannot be merged without it.

Solution

This can be avoided by using a helper function to do this work and keep things simple.

Are you willing to submit a PR?

  • [x] I'm willing to submit a PR!

SharonIV0x86 avatar Apr 27 '25 13:04 SharonIV0x86

@LindaSummer Your views on this?

SharonIV0x86 avatar Apr 27 '25 13:04 SharonIV0x86

Hi @SharonIV0x86 ,

Good idea! ❤

Thanks very much for your effort and sorry for my delay in past two weeks.

I think it is valuable to have a try, and it could be do after the command's implementation.

But it will be better to reserve for some specialized operations like the QUANTILE and CDF's requirements.

Best Regards, Edward

LindaSummer avatar Apr 28 '25 12:04 LindaSummer

I think it is valuable to have a try, and it could be do after the command's implementation.

Sure, will get back to this after the current PRs are merged related to the commands implementation

But it will be better to reserve for some specialized operations like the QUANTILE and CDF's requirements.

Yes, the operations like QUANTILE, CDF, RANK and REVRANK etc very much requires accessing the centroids and this refactor can be useful there.

SharonIV0x86 avatar Apr 28 '25 12:04 SharonIV0x86