neon
neon copied to clipboard
pageserver: don't put layers in the heatmap if they're probably going to be compacted-away soon
Compaction runs every 20 seconds, heatmap uploads intentionally less frequent than that. The idea of heatmaps is to enable secondary tenants to prime their cache with long-existing large data, not to try and race with delta layers pending compaction.
It is not an error to include short-lived layers, but it will waste resources on the secondary locations for tenants doing intensive writes.
Some kind of check during update of the heatmap should be sufficient, such as ignoring layers that are younger than a few compaction intervals.