dragonfly
dragonfly copied to clipboard
feat (tiering): implementing periodic defragmentation for second tier
fixes https://github.com/dragonflydb/dragonfly/issues/2433
This update introduces a new feature that regularly performs a defragmentation process by scanning "external keys". If a key's SSD page has a bin utilization lower than a certain threshold, all the keys' values will be loaded back into memory, and offloading will be rescheduled for each. This process aims to consolidate fragmented keys into a new block, which will be better utilized.
Todo: will move the defrag into a different place rather than abusing the heart beat function.
@adiholden For now, I did not scan the reference count table as that table doesn't provide the bin size ... of course, one can always grab a hash, get the iterator, then get the length, and then recalculate bin_size ... but I felt it is just a bit easier to directly obtain it from the PrimeIterator passed to the defrag function from db_slice.
please add unit tests to your code