[BUG] keydb 6.3.1 appendonly.aof becomes large fast

[Antiarchitect]

Describe the bug

Using https://artifacthub.io/packages/helm/enapter/keydb (Master - Master) mode I've noticed that one of the nodes makes appendonly.aof very large too quickly. Seems 6.2.2 was OK (but could be other factors affecting). This KeyDB setup is used as Redis replacement in Sentry setup.

To reproduce

Wait for some time?

Expected behavior

appendonly.aof files are small in size and relatively same size among the nodes

Additional information

Affected instance logs:

keydb 1:19:S 12 Jul 2022 13:12:06.344 * Background AOF buffer size: 3880 MB
keydb 1:19:S 12 Jul 2022 13:21:08.751 # Background AOF buffer size: 3980 MB
keydb 1:19:S 12 Jul 2022 13:30:11.026 * Background AOF buffer size: 4080 MB
keydb 1:19:S 12 Jul 2022 13:33:05.000 * Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down KeyDB.
keydb 1:19:S 12 Jul 2022 13:33:53.019 * Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down KeyDB.
keydb 1:19:S 12 Jul 2022 13:34:06.001 * Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down KeyDB.
keydb 1:19:S 12 Jul 2022 13:39:08.043 * Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down KeyDB.
keydb 1:19:S 12 Jul 2022 13:39:12.630 * Background AOF buffer size: 4180 MB
keydb 1:19:S 12 Jul 2022 13:40:53.019 * Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down KeyDB.
keydb 1:20:S 12 Jul 2022 13:40:56.033 * Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down KeyDB.
keydb 1:20:S 12 Jul 2022 13:41:03.054 * Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down KeyDB.
keydb 1:20:S 12 Jul 2022 13:43:11.055 * Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down KeyDB.
keydb 1:20:S 12 Jul 2022 13:43:43.051 * Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down KeyDB.
keydb 1:19:S 12 Jul 2022 13:43:59.515 # Warning, detected child with unmatched pid: 32691
keydb 1:19:S 12 Jul 2022 13:43:59.616 # Warning, detected child with unmatched pid: 32700
keydb NOTICE: Detuning locks due to high load per core: 108.51%
keydb NOTICE: Detuning locks due to high load per core: 95.23%
keydb NOTICE: Detuning locks due to high load per core: 101.36%

[yammymushry]

We faced the same problem in our production server. Hope the fix comes out soon.

[Serpent6877]

Same issue here. Have a 5GB dump.rdb file but my appendonly.aof is quickly up to 205GB.

Using the default aof settings:

auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb

[msotheeswaran-sc]

This may be related to forkless background saving, if anyone is still experiencing this issue can you try disabling forkless background saving with config "use-fork yes"