PageCache warmup on Server Restarts and Segment Refresh
Summary
Issue link : https://github.com/apache/pinot/issues/15675
🚀 What this PR delivers
🔄 Typical flow
Create Query file
curl -X POST /pagecache/queries/airlineStats?tableType=OFFLINE \
-d '["SELECT …", "SELECT …"]' -H 'Content-Type: application/json'
2. Server restart
Startup hook calls warm-up for each enabled table; queries run for the configured warmup duration
3. Segment refresh
Controller triggers warm-up for only the newly uploaded segments using the consistent protocol feature.
Backward compatibility
Fully opt-in – no existing table is affected until pageCacheWarmupConfig is set. Query path remains unchanged; warm-up traffic stays on the secondary queue during refresh if BinaryWorkloadScheduler is enabled. During restart it uses the same primary queue since it is not competing with live traffic
Testing
unit test, manually tested using quick-starter and test actual servers
Codecov Report
:x: Patch coverage is 23.51695% with 361 lines in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 63.13%. Comparing base (62b20d9) to head (b19b478).
:warning: Report is 1 commits behind head on master.
Additional details and impacted files
@@ Coverage Diff @@
## master #16033 +/- ##
============================================
- Coverage 63.21% 63.13% -0.09%
+ Complexity 1428 1422 -6
============================================
Files 3120 3126 +6
Lines 184759 185228 +469
Branches 28320 28368 +48
============================================
+ Hits 116801 116947 +146
- Misses 58935 59246 +311
- Partials 9023 9035 +12
| Flag | Coverage Δ | |
|---|---|---|
| custom-integration1 | ? |
|
| integration | ? |
|
| integration1 | ? |
|
| integration2 | ? |
|
| java-11 | 63.11% <23.51%> (-0.07%) |
:arrow_down: |
| java-21 | 63.07% <23.51%> (-0.11%) |
:arrow_down: |
| temurin | 63.13% <23.51%> (-0.09%) |
:arrow_down: |
| unittests | 63.13% <23.51%> (-0.08%) |
:arrow_down: |
| unittests1 | 55.96% <29.26%> (+0.02%) |
:arrow_up: |
| unittests2 | 33.73% <23.51%> (+<0.01%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.