cdc region cache load store failed

[3AceShowHand]

a huge number of logs from the RegionCache:

cdc version v5.3.1, we shall take investigation into the RegionCache

[2022/08/06 20:22:25.617 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.618 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.630 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=4] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.631 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.631 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=1] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.631 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.633 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=4] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.634 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.639 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=7] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.639 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.642 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=7] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.642 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.650 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=1] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.650 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.651 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=4] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.651 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.652 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=4] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.652 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.653 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=4] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.653 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.664 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=4] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.664 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.664 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=4] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.664 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.668 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=4] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.668 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.673 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=4] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]
[2022/08/06 20:22:25.673 +08:00] [ERROR] [error.go:276] ["encountered error"] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"] [stack="github.com/tikv/client-go/v2/error.Log\n\tgithub.com/tikv/client-go/[email protected]/error/error.go:276\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:438\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406"]
[2022/08/06 20:22:25.681 +08:00] [ERROR] [region_cache.go:1958] ["loadStore from PD failed"] [id=4] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"] [errorVerbose="rpc error: code = DeadlineExceeded desc = context deadline exceeded\ngithub.com/tikv/pd/client.(*client).GetStore\n\tgithub.com/tikv/[email protected]/client/client.go:1479\ngithub.com/tikv/client-go/v2/internal/locate.(*Store).reResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:1948\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).checkAndResolve\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:437\ngithub.com/tikv/client-go/v2/internal/locate.(*RegionCache).asyncCheckAndResolveLoop\n\tgithub.com/tikv/client-go/[email protected]/internal/locate/region_cache.go:406\nruntime.goexit\n\truntime/asm_amd64.s:1371"]