RedisShake
RedisShake copied to clipboard
sync同步报EOF问题
- [x] 请确保已经看过 wiki:https://github.com/alibaba/RedisShake/wiki
- [x] 请确保已经学习过 Markdown 语法,良好的排版有助于维护人员了解你的问题
- [ ] 请在此提供足够的信息供社区维护人员排查问题
- [ ] 请在提交 issue 前删除此模板中多余的文字,包括这几句话
问题描述 报过两次connection reset by peer redis ,后面两次同步报EOF,不知道为什么?这里贴出配置信息
type = "sync"
[source]
version = 5.0 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ...
address = "xxxx:6379"
username = "" # keep empty if not using ACL
password = "" # keep empty if no authentication is required
tls = false
elasticache_psync = "" # using when source is ElastiCache. ref: https://github.com/alibaba/RedisShake/issues/373
[target]
type = "standalone" # "standalone" or "cluster"
version = 6.2 # redis version, such as 2.8, 4.0, 5.0, 6.0, 6.2, 7.0, ...
address = "xxxx:6379"
username = "" # keep empty if not using ACL
password = "" # keep empty if no authentication is required
tls = false
[advanced]
dir = "data"
ncpu = 4
pprof_port = 0
metrics_port = 0
log_file = "redis-shake.log"
log_level = "debug" # debug, info or warn
log_interval = 3 # in seconds
rdb_restore_command_behavior = "rewrite" # panic, rewrite or skip
pipeline_count_limit = 1024
#此处开始报错,我改了7000_000_000,仍然报错
target_redis_client_max_querybuf_len = 7000_000_000
target_redis_proto_max_bulk_len = 512_000_000
redis-shake 的日志:
{"level":"info","time":"2023-05-09T20:30:29+08:00","message":"source db bgsave finished. timeUsed=[227.15]s, address=[8.143.197.105:6379]"}
{"level":"info","time":"2023-05-09T20:30:29+08:00","message":"received rdb length. length=[7313790769]"}
{"level":"info","time":"2023-05-09T20:30:29+08:00","message":"create dump.rdb file. filename_path=[dump.rdb]"}
{"level":"info","time":"2023-05-09T20:30:32+08:00","message":"syncing rdb. percent=[0.00]%, allowOps=[0.00], disallowOps=[0.00], entryId=[0], InQueueEntriesCount=[0], unansweredBytesCount=[0]bytes, rdbFileSize=[6.811]G, rdbSendSize=[0.000]G"}
{"level":"info","time":"2023-05-09T20:30:35+08:00","message":"syncing rdb. percent=[0.00]%, allowOps=[0.00], disallowOps=[0.00], entryId=[0], InQueueEntriesCount=[0], unansweredBytesCount=[0]bytes, rdbFileSize=[6.811]G, rdbSendSize=[0.000]G"}
{"level":"info","time":"2023-05-09T20:49:44+08:00","message":"syncing rdb. percent=[0.00]%, allowOps=[0.00], disallowOps=[0.00], entryId=[0], InQueueEntriesCount=[0], unansweredBytesCount=[0]bytes, rdbFileSize=[6.811]G, rdbSendSize=[0.000]G"}
{"level":"info","time":"2023-05-09T20:49:45+08:00","message":"goroutine 33 [running]: [runtime/debug.Stack()]<-runtime/debug/stack.go:24 +0x65 [github.com/alibaba/RedisShake/internal/log.Panicf({0x79a4ac, 0x3}, {0x0, 0x0, 0x0})]<-github.com/alibaba/RedisShake/internal/log/func.go:27 +0x36 [github.com/alibaba/RedisShake/internal/log.PanicError({0x8167a0, 0xc0000a6060})]<-github.com/alibaba/RedisShake/internal/log/func.go:36 +0x33 [github.com/alibaba/RedisShake/internal/reader.(*psyncReader).saveRDB(0xc000072000)]<-github.com/alibaba/RedisShake/internal/reader/psync.go:177 +0x9ff [github.com/alibaba/RedisShake/internal/reader.(*psyncReader).StartRead.func1()]<-github.com/alibaba/RedisShake/internal/reader/psync.go:46 +0x72 [created by github.com/alibaba/RedisShake/internal/reader.(*psyncReader).StartRead]<-github.com/alibaba/RedisShake/internal/reader/psync.go:43 +0x8f ["}
{"level":"panic","time":"2023-05-09T20:49:45+08:00","message":"EOF"}
源端 Redis 版本:5 (自建) 日志:
暂无
目的端 Redis 版本:6(自建) 日志:
暂无
请提供源端 Redis 日志
1535:M 09 May 2023 20:45:11.427 * Background saving terminated with success 1535:M 09 May 2023 20:46:12.020 * 10000 changes in 60 seconds. Saving... 1535:M 09 May 2023 20:46:12.600 * Background saving started by pid 8534 1535:M 09 May 2023 20:46:28.000 # Client id=3494869 addr=xxxx:53032 fd=1234 name= age=1294 idle=1 flags=S db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=16381 oll=13915 omem=285583496 events=rw cmd=replconf scheduled to be closed ASAP for overcoming of output buffer limits. 1535:M 09 May 2023 20:46:28.020 # Connection with replica xxxx:10007 lost. 8534:C 09 May 2023 20:50:00.162 * DB saved on disk 8534:C 09 May 2023 20:50:00.656 * RDB: 3765 MB of memory used by copy-on-write 1535:M 09 May 2023 20:50:03.262 * Background saving terminated with success
“scheduled to be closed ASAP for overcoming of output buffer limits” 源端断开的连接,网上搜一下这个报错,有很多原理解释。
config set client-output-buffer-limit "slave 0 0 0" 源redis设置这个,可以解决。又遇到 write: connection reset by peer;还在继续找问题
不要改 target_redis_client_max_querybuf_len
改回原始了,仍然报写入连接失败,换方案了
@IJustOne 请问新方案用什么做的?
@IJustOne 老哥 这个问题怎么解决的呀 tks