ozone
ozone copied to clipboard
HDDS-8784. trigger compaction outside of volume check.
What changes were proposed in this pull request?
Currently RocksDB compaction is triggered for schema v3 RocksDBs (one DB per volume) as part of the volume check. Periodic manual compaction is necessary because import of sst files as part of container replication can lead to many small sst files in RocksDB.
This operation may slow down the volume check and can have unintended consequences if the minimum gap between volume checks is set very high or very low. Ideally compaction should be triggered on its own independently controlled thread.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8784
How was this patch tested?
UT: org.apache.hadoop.ozone.container.keyvalue.TestKeyValueContainer#testAutoCompactionSmallSstFile
Thanks @guohao-rosicky for working on this. It looks like several tests are failing, please check.
https://github.com/guohao-rosicky/ozone/actions/runs/8892159377/job/24417356838#step:5:1252
https://github.com/guohao-rosicky/ozone/actions/runs/8892159377/job/24417354447#step:5:2662
https://github.com/guohao-rosicky/ozone/actions/runs/8892159377/job/24417354447#step:5:2875
https://github.com/guohao-rosicky/ozone/actions/runs/8892159377/job/24417357696#step:5:2144
Change seems fine. Will wait for the metrics questions to get answered and then give it a try.
I have already added the metric @kerneltime .
LGTM + 1. Thanks @guohao-rosicky for the contribution, @kerneltime for the review.