RedisShake
RedisShake copied to clipboard
slave buffer on source redis server is growing indefinitely
Issue Description:
The slave client buffer keep growing on the source redis, even after RDB sync finished. There is no any errors in source or detination, the only issue is that the client buffer growing indefinitely and I've concern that it will cause out of memory soon in the source redis server.
Is this normal, anything I can do about it?
Environment
- RedisShake version: v3.1.11
- Redis source: 6.2
- Redis destination: 6.2
- Redis sentinel
- Deployed on Cloud Provider: No
Logs
Last logs
2023-10-27 17:27:12 INF syncing aof. allowOps=[6382.40], disallowOps=[15.00], entryId=[30404975], InQueueEntriesCount=[0], unansweredBytesCount=[54076]bytes, diff=[87537], aofReceivedOffset=[29786229765029], aofAppliedOffset=[29786229677492]
2023-10-27 17:27:17 INF syncing aof. allowOps=[4823.60], disallowOps=[10.80], entryId=[30429147], InQueueEntriesCount=[0], unansweredBytesCount=[0]bytes, diff=[114688], aofReceivedOffset=[29786240152485], aofAppliedOffset=[29786240037797]
2023-10-27 17:27:22 INF syncing aof. allowOps=[5342.00], disallowOps=[11.60], entryId=[30455915], InQueueEntriesCount=[0], unansweredBytesCount=[52487]bytes, diff=[67009], aofReceivedOffset=[29786249573285], aofAppliedOffset=[29786249506276]
2023-10-27 17:27:27 INF syncing aof. allowOps=[6856.20], disallowOps=[9.20], entryId=[30490242], InQueueEntriesCount=[0], unansweredBytesCount=[56608]bytes, diff=[90112], aofReceivedOffset=[29786259993509], aofAppliedOffset=[29786259903397]
2023-10-27 17:27:32 INF syncing aof. allowOps=[4987.20], disallowOps=[10.40], entryId=[30515230], InQueueEntriesCount=[0], unansweredBytesCount=[55174]bytes, diff=[69636], aofReceivedOffset=[29786269889445], aofAppliedOffset=[29786269819809]
Additional Information:
Graph
Configuation
type = "sync"
[source]
version = 6.2
address = "slave.source.consul:6379"
username = ""
password = ""
tls = false
[target]
type = "standalone" #
version = 6.2
address = "master.destination.consul:6379"
username = ""
password = ""
tls = false
[advanced]
dir = "data"
# runtime.GOMAXPROCS, 0 means use runtime.NumCPU() cpu cores
ncpu = 4
# pprof port, 0 means disable
pprof_port = 0
# metric port, 0 means disable
metrics_port = 0
# log
log_file = "redis-shake.log"
log_level = "info"
log_interval = 5
rdb_restore_command_behavior = "rewrite"
# pipeline
pipeline_count_limit = 1024
target_redis_client_max_querybuf_len = 1024_000_000
target_redis_proto_max_bulk_len = 512_000_000
Redis shake also configured with this filter:
-- pubsub: skip PUBLISH command
function filter(id, is_base, group, cmd_name, keys, slots, db_id, timestamp_ms)
if cmd_name == "PUBLISH" then
return 1, db_id -- disallow
end
return 0, db_id -- allow
end
I have no idea. There was a similar issue(#602) but it should have been fixed.
- Please provide more info information through the
info all
command. - Try v4.0, it may help.
The issue is on source redis not destination. Anyway, I already finished running redis shake and migrated all the data, so not an issue for me anymore, but maybe others will experience the issue as well, feel free to close the issue