KeyDB icon indicating copy to clipboard operation
KeyDB copied to clipboard

[NEW] Throttle TTL cleanup CPU usage from EXPIREMEMBER to protect the server

Open Zeg0 opened this issue 9 months ago • 3 comments

The problem/use-case that the feature addresses

I am running a keydb 6.3.4. I turned off all persistence and ran it in ram only. I insert a few thousand unsorted sets (SADD setname objectname). Inside each set can be a few thousand objects. Therefore i have about 300000 to a million objects. When i issue EXPIREMEMBER setname objectname 14400 (in 4hrs) i usually get some CPU spikes within 4 hours. About 4-5 times a day these CPU spikes get so large, that the keydb server stops handling normal connections, causing i/o timeout or pool timeout on the client side for 1-5 minutes.

Description of the feature

I would like to have an option to throttle the EXPIREMEMBER cleanup job to a maximum CPU usage so i can protect the core funtionality of the keydb server. It is not mandatory that the keys expire in exactly the given time, it is way more important that i can insert new keys to the server.

Alternatives you've considered

Alternatively i would remove the EXPIREMEMBER from my application and replace the unsorted Set (SADD) with a sorted set (ZADD) with a score=now.unix_time and make another redis client that functions as a cleanup job (ZREMRANGEBYSCORE) through a cronjob that is running when my server has low CPU load. For example nightly.

Zeg0 avatar Mar 21 '25 11:03 Zeg0