Feature:Online Query and Dynamic Modification of Table-level RocksDB Options

[ruojieranyishen]

Background

Currently, in the Pegasus 2.4.1 version, dynamic querying and modification of RocksDB options at the table level are not supported. While the Pegasus shell provides commands like set_app_envs and create for configuring certain table-level options, it does not include RocksDB options. Consequently, adding the configure table-level RocksDB options ability would enhance the app configuration flexibility in Pegasus.

Goals

This issue aims to achieve the following goals:

1. Provide online modification functionality for table-level RocksDB configuration.
2. Provide online query functionality for table-level RocksDB configuration.
3. Research on the principles, interdependencies, and modification risks of the rocksdb options used in Pegasus with rocksdb-6.6.4 version, and compile a comprehensive documentation on the usage of table-level rocksdb options.
4. By modifying the logic of the Usage Scenario feature, unify the approach for setting table-level RocksDB options.

Solution

Here are the solutions for goals 1 and 2. The storage location of rocksdb option is as follows:

1. Utilize the config.ini file to provide default global RocksDB options information.
2. Store table-specific RocksDB options in the envs section of the app_info in ZooKeeper (zk).

Three scenarios to set specific rocksdb option, and the solution as follows：

1. [Create a replica with specific rocksdb option] Set specify option by create appname -e rocksdb.options command If a replica is not created, use the app envs rocksdb option from metaserver to create. For a already created replica, because the app envs is not transmitted from the metaserver, use options from the RocksDB OPTION file as the reference.

2. [Modify a specific rocksdb option of an online replica] Set specify option by set_app_envs rocksdb.options command

3. [Query the modified option online] Get specify option by get_app_envs command

To validate the feasibility of the proposed solution, I will use rocksdb.write_buffer_size as a dynamically modifiable parameter and rocksdb.num_levels as a non-dynamically modifiable parameter to test my idea.

[ruojieranyishen]

This is the app creation process, I show the app envs passing through the num_levels option.

[ruojieranyishen]

This is the specific creation process of replica, including my two modifications. Respectively deal with creating a new rocksdb and restarting rocksdb.

Create a replica key process：

1. new replica ： Create a replica, which contains env information num_levels=5, and the actual rocksdb has not been created
2. store_app_info(_app_info) ： Persist app_info information to .app_info file
3. _app.reset(replication_app_base::new_storage_instance(_app_info.app_type, this)); ：Construct pegasus_server_impl. Parameter this pointer is replica*, which contains env information num_levels=5.
4. pegasus_server_impl::pegasus_server_impl ：Construct pegasus_server_impl. Initialize rocksdb option via FLAGS_rocksdb_*
5. update_app_envs_before_open_db(envs); ：Make the env information effective before creating the rocksdb instance.
6. reset_usage_scenario_options ： Load rocksdb option from OPTION-XXXXX file.

[ruojieranyishen]

I show the set_app_envs procedure using the write_buffer_size option. My third modification solves the online modification of dynamic rocksdb options.