pinot icon indicating copy to clipboard operation
pinot copied to clipboard

Rebuild BloomFilter index on segment reload if bloom filter config is updated

Open anilsomisetty opened this issue 1 month ago • 3 comments

This PR adds the feature changes described in issue: https://github.com/apache/pinot/issues/17137

Changes:

  • Stored fpp value in the header of bloomFilter index as metadata
  • Set the bloom filter version to 2 to achieve above and to maintain backward compatibility
  • Changes to rebuild bloom filter index on it's config change and segment reload

anilsomisetty avatar Dec 04 '25 11:12 anilsomisetty

@xiangfu0 @yashmayya could you please review the pr ?

anilsomisetty avatar Dec 04 '25 11:12 anilsomisetty

cc @swaminathanmanish @KKcorps @abhishekbafna

Jackie-Jiang avatar Dec 09 '25 02:12 Jackie-Jiang

Codecov Report

:x: Patch coverage is 78.33333% with 13 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 63.32%. Comparing base (8bfb907) to head (22a3a1a). :warning: Report is 14 commits behind head on master.

Files with missing lines Patch % Lines
...t/index/loader/bloomfilter/BloomFilterHandler.java 67.56% 8 Missing and 4 partials :warning:
.../index/readers/bloom/BloomFilterReaderFactory.java 88.88% 0 Missing and 1 partial :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17320      +/-   ##
============================================
+ Coverage     63.24%   63.32%   +0.08%     
- Complexity     1424     1433       +9     
============================================
  Files          3133     3133              
  Lines        186150   186216      +66     
  Branches      28412    28420       +8     
============================================
+ Hits         117737   117930     +193     
+ Misses        59351    59199     -152     
- Partials       9062     9087      +25     
Flag Coverage Δ
custom-integration1 100.00% <ø> (?)
integration 100.00% <ø> (+100.00%) :arrow_up:
integration1 100.00% <ø> (?)
integration2 0.00% <ø> (ø)
java-11 63.23% <78.33%> (+<0.01%) :arrow_up:
java-21 63.29% <78.33%> (+29.41%) :arrow_up:
temurin 63.32% <78.33%> (+0.08%) :arrow_up:
unittests 63.32% <78.33%> (+0.07%) :arrow_up:
unittests1 55.64% <41.66%> (-0.01%) :arrow_down:
unittests2 33.98% <78.33%> (+0.09%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov-commenter avatar Dec 09 '25 03:12 codecov-commenter