bookkeeper icon indicating copy to clipboard operation
bookkeeper copied to clipboard

Prioritize compaction of entry logs with the lowest amount of remaining usable data

Open dlg99 opened this issue 3 years ago • 0 comments

FEATURE REQUEST

  1. Please describe the feature you are requesting.

Prioritize compaction to free up more space faster.

  1. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have).

must-have

  1. Provide any additional detail on your proposed use case for this feature.

Looking at GarbageCollectorThread:

doCompactEntryLogs() iterates over entry logs in whatever natural order they happen to be, picks the first with usage below thresholds and starts compacting. Do major compaction it means we can start compaction with an entry log at 80% utilization instead of e.g. one with 10%. This can be easily fixed by building a PriorityQueue of entry logs, ordering by lowest utilization (meta.getUsage()) to free up more space sooner. Building of the queue should not take too much time and can be combined with doGcEntryLogs() which iterates over all entries in entryLogMetaMap anyway; memory-wise it should be fine too.

dlg99 avatar Jul 06 '22 18:07 dlg99