incubator-pegasus icon indicating copy to clipboard operation
incubator-pegasus copied to clipboard

fix: rocksdb options not changed even if update in Pegasus config file

Open WHBANG opened this issue 3 years ago • 0 comments

What problem does this PR solve?

https://github.com/apache/incubator-pegasus/issues/1025

What is changed and how does it work?

The specific reason is that if you open an existing rocksdb, the usage scenario will be set to normal first, and then set to the corresponding mode according to the actual usage scenario. This process will change the relevant options, causing rocksdb to execute flush and compact, so in order to avoid this situation (https://github.com/apache/incubator-pegasus/pull/587), rocksdb::LoadLatestOptions is called, and these parameters are initialized with the last value, so the last thing you see is that if the configuration file is modified, and without changing the usage scenario mode, these modified configurations will not take effect after restart.

Options involved:

level0_file_num_compaction_trigger
level0_slowdown_writes_trigger
level0_stop_writes_trigger
soft_pending_compaction_bytes_limit
hard_pending_compaction_bytes_limit
disable_auto_compactions
max_compaction_bytes
write_buffer_size
max_write_buffer_number

Solution:

  1. After opening rocksdb, calculate the value of the corresponding options according to the usage scenario mode in the environment variable, compare whether there is a change, and call setOpion() if there is a change to set the newly calculated value.
  2. If the usage scenario mode has been changed during this process, the options will be the correct value, and nothing needs to be done.
Tests
  • Unit test

  • Manual test (add detailed scripts or steps below)

WHBANG avatar Aug 09 '22 13:08 WHBANG