kvrocks icon indicating copy to clipboard operation
kvrocks copied to clipboard

Customize prefix extractor for subkey

Open shangxiaoxiong opened this issue 3 years ago • 11 comments

Customize prefix extractor for subkey. Range query prefermance is likely improved when applying this prefix extractor to subkey column family. It is more effective than FixedPrefixTransform and CappedPrefixTransform provided by Rocksdb. https://github.com/apache/incubator-kvrocks/pull/508 shows usage of prefix extractor.

shangxiaoxiong avatar Sep 23 '22 03:09 shangxiaoxiong

Cool!How much performance improvement did you get from testing?

caipengbo avatar Sep 23 '22 03:09 caipengbo

l did an incomplete test. For 1 millon times short range query, the query time is reduced more than 20%.

shangxiaoxiong avatar Sep 23 '22 03:09 shangxiaoxiong

@shangxiaoxiong I didn't see where's using this prefix extractor.

git-hulk avatar Sep 23 '22 05:09 git-hulk

https://github.com/apache/incubator-kvrocks/pull/508 shows the usage of prefix extractor (FixedPrefixTransform provided by Rocksdb) @git-hulk.

shangxiaoxiong avatar Sep 23 '22 05:09 shangxiaoxiong

@shangxiaoxiong So I think we can merge what #508 does into this PR and use the new prefix extractor.

git-hulk avatar Sep 23 '22 05:09 git-hulk

@shangxiaoxiong So I think we can merge what #508 does into this PR and use the new prefix extractor.

l think so and l can merge https://github.com/apache/incubator-kvrocks/pull/508 if there is no problem about the new prefix extractor.

shangxiaoxiong avatar Sep 23 '22 05:09 shangxiaoxiong

Got it, thanks a lot.

git-hulk avatar Sep 23 '22 05:09 git-hulk

Thank you for your contribution, it would be better if you could explain it in detail in this PR, which can help us to see the code and understand the design.

caipengbo avatar Sep 23 '22 06:09 caipengbo

Thank you for your contribution, it would be better if you could explain it in detail in this PR, which can help us to see the code and understand the design.

Don't make me hammer you.

shangxiaoxiong avatar Sep 23 '22 06:09 shangxiaoxiong

@caipengbo is right, we can add more informations to make this easier to review.

git-hulk avatar Sep 23 '22 07:09 git-hulk

@caipengbo is right, we can add more informations to make this easier to review.

ha ha ha ! On the way.

shangxiaoxiong avatar Sep 23 '22 07:09 shangxiaoxiong

@shangxiaoxiong You can help to pull the @ShooterIT's PR into this, then we can pull others to have a look again.

git-hulk avatar Sep 26 '22 15:09 git-hulk

@shangxiaoxiong You can help to pull the @ShooterIT's PR into this, then we can pull others to have a look again.

OK

shangxiaoxiong avatar Sep 27 '22 06:09 shangxiaoxiong

Hi @shangxiaoxiong! Any progress on this pull request?

tisonkun avatar Nov 08 '22 08:11 tisonkun

Hi @shangxiaoxiong , do you still have time to continue this PR?

PragmaTwice avatar Mar 23 '23 05:03 PragmaTwice

@PragmaTwice I will continue working on this and also #508 recently if @shangxiaoxiong and @ShooterIT don't have time to move on.

git-hulk avatar Mar 28 '23 10:03 git-hulk

I will raise another PR to do this.

git-hulk avatar Apr 03 '23 11:04 git-hulk