Tendis icon indicating copy to clipboard operation
Tendis copied to clipboard

Avoid reading non-meta keys to invoke compaction_filter during compaction

Open leonliao opened this issue 1 year ago • 1 comments

Description

Tendis saves data's TTL in the value of RT_DATA_META-type keys ONLY . And it utilizes the RocksDB's compaction_filter to read TTL out of value and then checks whether the key is expired.

So it is not necessary to read blob value of non RT_DATA_META-typed keys. Reading blob_values out of RocksDB consumes too much read-IO bandwidth.

Expected Behavior

Only reads the value of RT_DATA_META-type keys

Current Behavior

Reads all value of all type of keys.

Possible Solution

Use the FilterBlobByKey of RocksDB's blobdb feature, checks whether it is necessary to read blob value.

Context

The recommended solution saves the disk read IO bandwidth significantly when using set/hash with "big" values (>min_blob_size) when enabling RocksDB's enable_blob_files.

leonliao avatar Apr 17 '23 10:04 leonliao

tendis2.7.0版本参照leonliao同学的pr和建议,对此问题进行了优化,再次感谢~

takenliu avatar Jan 31 '24 02:01 takenliu