gitlab-ci-pipelines-exporter
gitlab-ci-pipelines-exporter copied to clipboard
unstable k8s pod due to redis issue
Hi, I have deployed gitlab-ci-exporter helm chart with redis HA but after scheduling metrics step i have freceived exception which is repeating on any setup of redis (standalone, or replication).
` {"environments-count":0,"level":"info","msg":"scheduling metrics pull","refs-count":127,"time":"2021-06-23T12:11:49Z"} {"environments-count":0,"level":"info","msg":"scheduling metrics pull","refs-count":127,"time":"2021-06-23T12:12:34Z"} {"environments-count":0,"level":"info","msg":"scheduling metrics pull","refs-count":127,"time":"2021-06-23T12:12:35Z"} {"environments-count":0,"level":"info","msg":"scheduling metrics pull","refs-count":127,"time":"2021-06-23T12:12:35Z"} {"environments-count":0,"level":"info","msg":"scheduling metrics pull","refs-count":127,"time":"2021-06-23T12:12:37Z"} {"environments-count":0,"level":"info","msg":"scheduling metrics pull","refs-count":127,"time":"2021-06-23T12:12:37Z"} {"environments-count":0,"level":"info","msg":"scheduling metrics pull","refs-count":127,"time":"2021-06-23T12:19:28Z"} {"environments-count":0,"level":"info","msg":"scheduling metrics pull","refs-count":127,"time":"2021-06-23T12:19:31Z"} panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x87d1f7]
goroutine 53 [running]: github.com/go-redis/redis/v8/internal/pool.(*ConnPool).waitTurn(0xc000384280, 0x0, 0x0, 0x7fa61b6f8c88, 0x0) github.com/go-redis/redis/[email protected]/internal/pool/pool.go:273 +0x37 github.com/go-redis/redis/v8/internal/pool.(*ConnPool).Get(0xc000384280, 0x0, 0x0, 0xc000663830, 0x87a882, 0x123dbf0) github.com/go-redis/redis/[email protected]/internal/pool/pool.go:234 +0x52 github.com/go-redis/redis/v8.(*baseClient)._getConn(0xc000496cc0, 0x0, 0x0, 0x7fa61b6f8c88, 0xc000663898, 0x40db9b) github.com/go-redis/redis/[email protected]/redis.go:207 +0x4c github.com/go-redis/redis/v8.(*baseClient).getConn(0xc000496cc0, 0x0, 0x0, 0xf, 0x0, 0x0) github.com/go-redis/redis/[email protected]/redis.go:195 +0x66 github.com/go-redis/redis/v8.(*baseClient).withConn(0xc000496cc0, 0x0, 0x0, 0xc002674b10, 0x0, 0x0) github.com/go-redis/redis/[email protected]/redis.go:293 +0x127 github.com/go-redis/redis/v8.(*baseClient)._generalProcessPipeline(0xc000496cc0, 0x0, 0x0, 0xc0009e3640, 0x4, 0x4, 0xc0005f09a0, 0x10, 0xbddd40) github.com/go-redis/redis/[email protected]/redis.go:449 +0x113 github.com/go-redis/redis/v8.(*baseClient).generalProcessPipeline(0xc000496cc0, 0x0, 0x0, 0xc0009e3640, 0x4, 0x4, 0xc0005f09a0, 0x40db9b, 0xc00056fc80) github.com/go-redis/redis/[email protected]/redis.go:429 +0x77 github.com/go-redis/redis/v8.(*baseClient).processTxPipeline(...) github.com/go-redis/redis/[email protected]/redis.go:421 github.com/go-redis/redis/v8.hooks.processPipeline.func1(...) github.com/go-redis/redis/[email protected]/redis.go:94 github.com/go-redis/redis/v8.hooks.withContext(...) github.com/go-redis/redis/[email protected]/redis.go:133 github.com/go-redis/redis/v8.hooks.processPipeline(0x0, 0x0, 0x0, 0x0, 0x0, 0xc0009e3640, 0x4, 0x4, 0xc000663ce0, 0x8, ...) github.com/go-redis/redis/[email protected]/redis.go:93 +0x393 github.com/go-redis/redis/v8.hooks.processTxPipeline(0x0, 0x0, 0x0, 0x0, 0x0, 0xc0004960c0, 0x2, 0x2, 0xc000663ce0, 0x0, ...) github.com/go-redis/redis/[email protected]/redis.go:129 +0x2b7 github.com/go-redis/redis/v8.(*Client).processTxPipeline(...) github.com/go-redis/redis/[email protected]/redis.go:629 github.com/go-redis/redis/v8.(*Pipeline).Exec(0xc00049e1e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) github.com/go-redis/redis/[email protected]/pipeline.go:115 +0x102 github.com/vmihailenco/taskq/v3/redisq.(*Queue).Release(0xc0002fc000, 0xc000149600, 0x0, 0x1) github.com/vmihailenco/taskq/[email protected]/redisq/queue.go:232 +0x1ae github.com/vmihailenco/taskq/v3.(*Consumer).fetchMessages(0xc000128600, 0xd94408, 0xc0000aa018, 0xc00006a280, 0x3edd410c00, 0x0, 0x0, 0x0) github.com/vmihailenco/taskq/[email protected]/consumer.go:438 +0x276 github.com/vmihailenco/taskq/v3.(*Consumer).fetcher(0xc000128600, 0xd94408, 0xc0000aa018, 0x0) github.com/vmihailenco/taskq/[email protected]/consumer.go:393 +0x1c5 github.com/vmihailenco/taskq/v3.(*Consumer).addFetcher.func1(0xc000128600, 0xd94408, 0xc0000aa018, 0xc000000000) github.com/vmihailenco/taskq/[email protected]/consumer.go:294 +0x74 created by github.com/vmihailenco/taskq/v3.(*Consumer).addFetcher github.com/vmihailenco/taskq/[email protected]/consumer.go:292 +0xd3 `