kubernetes-replicator
kubernetes-replicator copied to clipboard
fatal error: concurrent map iteration and map write dealing with short-living namespaces
Describe the bug
Hi, we have the following config to replicate a docker private registry key across ~500 namespaces.
apiVersion: v1
kind: Secret
metadata:
name: docker-cred-secret
annotations:
replicator.v1.mittwald.de/replicate-to: "*"
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: ${docker_config_json}
Namespaces are dynamically created and removed on regular basis. A pod running kubernetes-replicator is restarting hourly or more often due to
fatal error: concurrent map iteration and map write
To Reproduce Create the above spec Start 250 static namespaces Start creating followed by termination 250 more namespaces constantly. Some namespaces live 10 seconds or less, some - not more than 1h. Aim is having a process of minimum 5 simultaneously namespaces termination
Expected behavior kubernetes-replicator works fine without restarts
Environment:
- Kubernetes version: 1.20
- kubernetes-replicator version: v2.7.3
Additional context
logs of failed pod:
time="2022-09-15T12:43:02Z" level=info msg="Secret replicator/docker-cred-secret to be replicated to: [*]" kind=Secret source=replicator/docker-cred-secret
time="2022-09-15T12:43:02Z" level=info msg="Checking if auto-ns-40888236/docker-cred-secret exists? false" kind=Secret source=replicator/docker-cred-secret target=auto-ns-40888236/docker-cred-secret
time="2022-09-15T12:43:02Z" level=debug msg="Creating a new secret secret auto-ns-40888236/docker-cred-secret" kind=Secret source=replicator/docker-cred-secret target=auto-ns-40888236/docker-cred-secret
time="2022-09-15T12:43:02Z" level=info msg="Replicated replicator/docker-cred-secret to: auto-ns-40888236" source=replicator/docker-cred-secret
fatal error: concurrent map iteration and map write
goroutine 5280 [running]:
runtime.throw(0x17f2bd1, 0x26)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/panic.go:1117 +0x72 fp=0xc0070db550 sp=0xc0070db520 pc=0x4361f2
runtime.mapiternext(0xc0070db8a0)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/map.go:858 +0x54c fp=0xc0070db5d0 sp=0xc0070db550 pc=0x410aac
github.com/mittwald/kubernetes-replicator/replicate/common.(*GenericReplicator).NamespaceAdded(0xc00003d130, 0xc000676160)
/home/runner/work/kubernetes-replicator/kubernetes-replicator/replicate/common/generic-replicator.go:153 +0x208 fp=0xc0070dbfb8 sp=0xc0070db5d0 pc=0x146a6e8
github.com/mittwald/kubernetes-replicator/replicate/common.(*GenericReplicator).NamespaceAdded-fm(0xc000676160)
/home/runner/work/kubernetes-replicator/kubernetes-replicator/replicate/common/generic-replicator.go:151 +0x3d fp=0xc0070dbfd8 sp=0xc0070dbfb8 pc=0x14733fd
runtime.goexit()
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0070dbfe0 sp=0xc0070dbfd8 pc=0x46ba81
created by github.com/mittwald/kubernetes-replicator/replicate/common.(*NamespaceWatcher).create.func1.1
/home/runner/work/kubernetes-replicator/kubernetes-replicator/replicate/common/namespaces.go:40 +0x70
and so on (see a comment)
time="2022-09-15T12:43:02Z" level=info msg="Secret replicator/docker-cred-secret to be replicated to: [*]" kind=Secret source=replicator/docker-cred-secret
time="2022-09-15T12:43:02Z" level=info msg="Checking if auto-ns-40888236/docker-cred-secret exists? false" kind=Secret source=replicator/docker-cred-secret target=auto-ns-40888236/docker-cred-secret
time="2022-09-15T12:43:02Z" level=debug msg="Creating a new secret secret auto-ns-40888236/docker-cred-secret" kind=Secret source=replicator/docker-cred-secret target=auto-ns-40888236/docker-cred-secret
time="2022-09-15T12:43:02Z" level=info msg="Replicated replicator/docker-cred-secret to: auto-ns-40888236" source=replicator/docker-cred-secret
fatal error: concurrent map iteration and map write
goroutine 5280 [running]:
runtime.throw(0x17f2bd1, 0x26)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/panic.go:1117 +0x72 fp=0xc0070db550 sp=0xc0070db520 pc=0x4361f2
runtime.mapiternext(0xc0070db8a0)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/map.go:858 +0x54c fp=0xc0070db5d0 sp=0xc0070db550 pc=0x410aac
github.com/mittwald/kubernetes-replicator/replicate/common.(*GenericReplicator).NamespaceAdded(0xc00003d130, 0xc000676160)
/home/runner/work/kubernetes-replicator/kubernetes-replicator/replicate/common/generic-replicator.go:153 +0x208 fp=0xc0070dbfb8 sp=0xc0070db5d0 pc=0x146a6e8
github.com/mittwald/kubernetes-replicator/replicate/common.(*GenericReplicator).NamespaceAdded-fm(0xc000676160)
/home/runner/work/kubernetes-replicator/kubernetes-replicator/replicate/common/generic-replicator.go:151 +0x3d fp=0xc0070dbfd8 sp=0xc0070dbfb8 pc=0x14733fd
runtime.goexit()
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0070dbfe0 sp=0xc0070dbfd8 pc=0x46ba81
created by github.com/mittwald/kubernetes-replicator/replicate/common.(*NamespaceWatcher).create.func1.1
/home/runner/work/kubernetes-replicator/kubernetes-replicator/replicate/common/namespaces.go:40 +0x70
goroutine 1 [IO wait]:
internal/poll.runtime_pollWait(0x7eff83617358, 0x72, 0x0)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/netpoll.go:227 +0x55
internal/poll.(*pollDesc).wait(0xc000378818, 0x72, 0x0, 0x0, 0x17d3c23)
/opt/hostedtoolcache/go/1.16.10/x64/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
/opt/hostedtoolcache/go/1.16.10/x64/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc000378800, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.10/x64/src/internal/poll/fd_unix.go:401 +0x212
net.(*netFD).accept(0xc000378800, 0x74edff6a613bba7e, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.10/x64/src/net/fd_unix.go:172 +0x45
net.(*TCPListener).accept(0xc000398678, 0x63231dd5, 0xc0003fdcc8, 0x490546)
/opt/hostedtoolcache/go/1.16.10/x64/src/net/tcpsock_posix.go:139 +0x32
net.(*TCPListener).Accept(0xc000398678, 0xc0003fdd18, 0x18, 0xc000000180, 0x6c7bbb)
/opt/hostedtoolcache/go/1.16.10/x64/src/net/tcpsock.go:261 +0x65
net/http.(*Server).Serve(0xc0001b2460, 0x19b7100, 0xc000398678, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.10/x64/src/net/http/server.go:2961 +0x285
net/http.(*Server).ListenAndServe(0xc0001b2460, 0xc0001b2460, 0x8)
/opt/hostedtoolcache/go/1.16.10/x64/src/net/http/server.go:2890 +0xba
net/http.ListenAndServe(...)
/opt/hostedtoolcache/go/1.16.10/x64/src/net/http/server.go:3144
main.main()
/home/runner/work/kubernetes-replicator/kubernetes-replicator/main.go:103 +0x4ce
goroutine 6 [chan receive]:
k8s.io/klog/v2.(*loggingT).flushDaemon(0x23577c0)
/home/runner/go/pkg/mod/k8s.io/klog/[email protected]/klog.go:1169 +0x8b
created by k8s.io/klog/v2.init.0
/home/runner/go/pkg/mod/k8s.io/klog/[email protected]/klog.go:420 +0xdf
goroutine 7 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0003a1608, 0x275)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0003a15f8)
/opt/hostedtoolcache/go/1.16.10/x64/src/sync/cond.go:56 +0x99
k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop(0xc0003a15e0, 0xc0003bc870, 0x0, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/delta_fifo.go:525 +0x98
k8s.io/client-go/tools/cache.(*controller).processLoop(0xc0003aba70)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:183 +0x42
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000486f90)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000486f90, 0x198d8a0, 0xc00049c000, 0xc00048a001, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000486f90, 0x3b9aca00, 0x0, 0xc0003c2601, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(...)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90
k8s.io/client-go/tools/cache.(*controller).Run(0xc0003aba70, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:154 +0x2e5
created by github.com/mittwald/kubernetes-replicator/replicate/common.(*NamespaceWatcher).create.func1
/home/runner/work/kubernetes-replicator/kubernetes-replicator/replicate/common/namespaces.go:70 +0x33a
goroutine 8 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0003a1568, 0x1493)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0003a1558)
/opt/hostedtoolcache/go/1.16.10/x64/src/sync/cond.go:56 +0x99
k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop(0xc0003a1540, 0xc0003bc750, 0x0, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/delta_fifo.go:525 +0x98
k8s.io/client-go/tools/cache.(*controller).processLoop(0xc0003ab9e0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:183 +0x42
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000094f38)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc002b9bf38, 0x198d8a0, 0xc0004020f0, 0xc00040a001, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000094f38, 0x3b9aca00, 0x0, 0xc000315901, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(...)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90
k8s.io/client-go/tools/cache.(*controller).Run(0xc0003ab9e0, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:154 +0x2e5
github.com/mittwald/kubernetes-replicator/replicate/common.(*GenericReplicator).Run(0xc00003d130)
/home/runner/work/kubernetes-replicator/kubernetes-replicator/replicate/common/generic-replicator.go:146 +0x122
created by main.main
/home/runner/work/kubernetes-replicator/kubernetes-replicator/main.go:88 +0x24c
goroutine 9 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0003a16a8, 0x14f4)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0003a1698)
/opt/hostedtoolcache/go/1.16.10/x64/src/sync/cond.go:56 +0x99
k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop(0xc0003a1680, 0xc0003bcab0, 0x0, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/delta_fifo.go:525 +0x98
k8s.io/client-go/tools/cache.(*controller).processLoop(0xc0003abb00)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:183 +0x42
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000482f38)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0004aff38, 0x198d8a0, 0xc00049c6f0, 0xc00048a301, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000482f38, 0x3b9aca00, 0x0, 0xc000315a01, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(...)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90
k8s.io/client-go/tools/cache.(*controller).Run(0xc0003abb00, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:154 +0x2e5
github.com/mittwald/kubernetes-replicator/replicate/common.(*GenericReplicator).Run(0xc00003d1e0)
/home/runner/work/kubernetes-replicator/kubernetes-replicator/replicate/common/generic-replicator.go:146 +0x122
created by main.main
/home/runner/work/kubernetes-replicator/kubernetes-replicator/main.go:90 +0x272
goroutine 10 [sync.Cond.Wait, 15 minutes]:
sync.runtime_notifyListWait(0xc0003a1748, 0x1)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0003a1738)
/opt/hostedtoolcache/go/1.16.10/x64/src/sync/cond.go:56 +0x99
k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop(0xc0003a1720, 0xc0003bcc60, 0x0, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/delta_fifo.go:525 +0x98
k8s.io/client-go/tools/cache.(*controller).processLoop(0xc0003abb90)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:183 +0x42
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000488f38)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000b05f38, 0x198d8a0, 0xc00049c360, 0xc00048a101, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000488f38, 0x3b9aca00, 0x0, 0xc000315b01, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(...)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90
k8s.io/client-go/tools/cache.(*controller).Run(0xc0003abb90, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:154 +0x2e5
github.com/mittwald/kubernetes-replicator/replicate/common.(*GenericReplicator).Run(0xc00003d290)
/home/runner/work/kubernetes-replicator/kubernetes-replicator/replicate/common/generic-replicator.go:146 +0x122
created by main.main
/home/runner/work/kubernetes-replicator/kubernetes-replicator/main.go:92 +0x298
goroutine 11 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0003a17e8, 0x139)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0003a17d8)
/opt/hostedtoolcache/go/1.16.10/x64/src/sync/cond.go:56 +0x99
k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop(0xc0003a17c0, 0xc0003bce10, 0x0, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/delta_fifo.go:525 +0x98
k8s.io/client-go/tools/cache.(*controller).processLoop(0xc0003abc20)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:183 +0x42
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000098f38)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc003885f38, 0x198d8a0, 0xc0003bd0e0, 0xc0003c0101, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000098f38, 0x3b9aca00, 0x0, 0xc0003c2801, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(...)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90
k8s.io/client-go/tools/cache.(*controller).Run(0xc0003abc20, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:154 +0x2e5
github.com/mittwald/kubernetes-replicator/replicate/common.(*GenericReplicator).Run(0xc00003d340)
/home/runner/work/kubernetes-replicator/kubernetes-replicator/replicate/common/generic-replicator.go:146 +0x122
created by main.main
/home/runner/work/kubernetes-replicator/kubernetes-replicator/main.go:94 +0x2be
goroutine 17 [chan receive, 15 minutes]:
k8s.io/client-go/tools/cache.(*controller).Run.func1(0xc000092300, 0xc0003ab9e0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:130 +0x34
created by k8s.io/client-go/tools/cache.(*controller).Run
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:129 +0xa5
goroutine 18 [select]:
k8s.io/client-go/tools/cache.(*Reflector).watchHandler(0xc00041e000, 0xc0c0e524c76de459, 0xaec8ddea65, 0x23574a0, 0x19956c0, 0xc002a9ad00, 0xc002b99b68, 0xc0060e13e0, 0xc000092300, 0x0, ...)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:468 +0x165
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch(0xc00041e000, 0xc000092300, 0x0, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:428 +0x738
k8s.io/client-go/tools/cache.(*Reflector).Run.func1()
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:221 +0x38
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0004246c8)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc002b99ec8, 0x198d880, 0xc00040e050, 0x1, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
k8s.io/client-go/tools/cache.(*Reflector).Run(0xc00041e000, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:220 +0x1ed
k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1()
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:56 +0x2e
k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc00040a030, 0xc000410018)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:73 +0x51
created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:71 +0x65
goroutine 56 [select, 15 minutes]:
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch.func2(0xc00041e000, 0xc000092300, 0xc000093d40, 0xc0060e13e0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:373 +0x125
created by k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:367 +0x2ed
goroutine 33 [chan receive, 15 minutes]:
k8s.io/client-go/tools/cache.(*controller).Run.func1(0xc000092300, 0xc0003aba70)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:130 +0x34
created by k8s.io/client-go/tools/cache.(*controller).Run
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:129 +0xa5
goroutine 34 [select]:
k8s.io/client-go/tools/cache.(*Reflector).watchHandler(0xc000498000, 0xc0c0e537de1bb527, 0xc0917fb334, 0x23574a0, 0x19956c0, 0xc000776d40, 0xc001a21b68, 0xc000330480, 0xc000092300, 0x0, ...)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:468 +0x165
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch(0xc000498000, 0xc000092300, 0x0, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:428 +0x738
k8s.io/client-go/tools/cache.(*Reflector).Run.func1()
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:221 +0x38
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000064ec8)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc001a21ec8, 0x198d880, 0xc000494000, 0x1, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
k8s.io/client-go/tools/cache.(*Reflector).Run(0xc000498000, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:220 +0x1ed
k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1()
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:56 +0x2e
k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc00048a010, 0xc00049a000)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:73 +0x51
created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:71 +0x65
goroutine 84 [select, 15 minutes]:
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch.func2(0xc000498000, 0xc000092300, 0xc00042c3c0, 0xc000330480)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:373 +0x125
created by k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:367 +0x2ed
goroutine 12 [chan receive, 15 minutes]:
k8s.io/client-go/tools/cache.(*controller).Run.func1(0xc000092300, 0xc0003abc20)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:130 +0x34
created by k8s.io/client-go/tools/cache.(*controller).Run
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:129 +0xa5
goroutine 13 [select]:
k8s.io/client-go/tools/cache.(*Reflector).watchHandler(0xc0001b2540, 0xc0c0e53820a4bcd6, 0xc0cfa384e5, 0x23574a0, 0x19956c0, 0xc00a26a3c0, 0xc003889b68, 0xc000a16060, 0xc000092300, 0x0, ...)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:468 +0x165
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch(0xc0001b2540, 0xc000092300, 0x0, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:428 +0x738
k8s.io/client-go/tools/cache.(*Reflector).Run.func1()
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:221 +0x38
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00005f6c8)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc003889ec8, 0x198d880, 0xc0003c4280, 0x1, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
k8s.io/client-go/tools/cache.(*Reflector).Run(0xc0001b2540, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:220 +0x1ed
k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1()
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:56 +0x2e
k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc0003c0160, 0xc0003986a8)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:73 +0x51
created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:71 +0x65
goroutine 54 [select, 15 minutes]:
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch.func2(0xc0001b2540, 0xc000092300, 0xc000092ea0, 0xc000a16060)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:373 +0x125
created by k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:367 +0x2ed
goroutine 36 [chan receive, 15 minutes]:
k8s.io/client-go/tools/cache.(*controller).Run.func1(0xc000092300, 0xc0003abb90)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:130 +0x34
created by k8s.io/client-go/tools/cache.(*controller).Run
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:129 +0xa5
goroutine 37 [select, 6 minutes]:
k8s.io/client-go/tools/cache.(*Reflector).watchHandler(0xc0004980e0, 0xc0c0e4fa9f1e528a, 0x87875efe86, 0x23574a0, 0x19956c0, 0xc000777600, 0xc00010fb68, 0xc000884180, 0xc000092300, 0x0, ...)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:468 +0x165
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch(0xc0004980e0, 0xc000092300, 0x0, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:428 +0x738
k8s.io/client-go/tools/cache.(*Reflector).Run.func1()
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:221 +0x38
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000064ec8)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00010fec8, 0x198d880, 0xc0004940f0, 0x1, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
k8s.io/client-go/tools/cache.(*Reflector).Run(0xc0004980e0, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:220 +0x1ed
k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1()
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:56 +0x2e
k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc00048a1a0, 0xc00049a090)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:73 +0x51
created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:71 +0x65
goroutine 99 [select, 15 minutes]:
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch.func2(0xc0004980e0, 0xc000092300, 0xc000092900, 0xc000884180)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:373 +0x125
created by k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:367 +0x2ed
goroutine 41 [chan receive, 15 minutes]:
k8s.io/client-go/tools/cache.(*controller).Run.func1(0xc000092300, 0xc0003abb00)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:130 +0x34
created by k8s.io/client-go/tools/cache.(*controller).Run
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:129 +0xa5
goroutine 42 [select]:
k8s.io/client-go/tools/cache.(*Reflector).watchHandler(0xc0004981c0, 0xc0c0e552a4ecbac4, 0xd9820326c7, 0x23574a0, 0x19956c0, 0xc0009637c0, 0xc003aefb68, 0xc0062d1980, 0xc000092300, 0x0, ...)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:468 +0x165
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch(0xc0004981c0, 0xc000092300, 0x0, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:428 +0x738
k8s.io/client-go/tools/cache.(*Reflector).Run.func1()
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:221 +0x38
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000420ec8)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc003aefec8, 0x198d880, 0xc0004942d0, 0x1, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x9b
k8s.io/client-go/tools/cache.(*Reflector).Run(0xc0004981c0, 0xc000092300)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:220 +0x1ed
k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1()
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:56 +0x2e
k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1(0xc00048a320, 0xc00049a138)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:73 +0x51
created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:71 +0x65
goroutine 82 [IO wait]:
internal/poll.runtime_pollWait(0x7eff83617270, 0x72, 0xffffffffffffffff)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/netpoll.go:227 +0x55
internal/poll.(*pollDesc).wait(0xc000444118, 0x72, 0xbe00, 0xbee2, 0xffffffffffffffff)
/opt/hostedtoolcache/go/1.16.10/x64/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
/opt/hostedtoolcache/go/1.16.10/x64/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc000444100, 0xc008e98000, 0xbee2, 0xbee2, 0x0, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.10/x64/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc000444100, 0xc008e98000, 0xbee2, 0xbee2, 0x95f2, 0xc008e9a8eb, 0x5)
/opt/hostedtoolcache/go/1.16.10/x64/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc00042e038, 0xc008e98000, 0xbee2, 0xbee2, 0x0, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.10/x64/src/net/net.go:183 +0x91
crypto/tls.(*atLeastReader).Read(0xc0040d2e70, 0xc008e98000, 0xbee2, 0xbee2, 0x95f2, 0xc000448800, 0x0)
/opt/hostedtoolcache/go/1.16.10/x64/src/crypto/tls/conn.go:776 +0x63
bytes.(*Buffer).ReadFrom(0xc00044a5f8, 0x198bd00, 0xc0040d2e70, 0x40b6a5, 0x1629440, 0x1773600)
/opt/hostedtoolcache/go/1.16.10/x64/src/bytes/buffer.go:204 +0xbe
crypto/tls.(*Conn).readFromUntil(0xc00044a380, 0x198dd00, 0xc00042e038, 0x5, 0xc00042e038, 0x2877)
/opt/hostedtoolcache/go/1.16.10/x64/src/crypto/tls/conn.go:798 +0xf3
crypto/tls.(*Conn).readRecordOrCCS(0xc00044a380, 0x0, 0x0, 0x3)
/opt/hostedtoolcache/go/1.16.10/x64/src/crypto/tls/conn.go:605 +0x115
crypto/tls.(*Conn).readRecord(...)
/opt/hostedtoolcache/go/1.16.10/x64/src/crypto/tls/conn.go:573
crypto/tls.(*Conn).Read(0xc00044a380, 0xc000587000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/opt/hostedtoolcache/go/1.16.10/x64/src/crypto/tls/conn.go:1276 +0x165
bufio.(*Reader).Read(0xc0004e8d20, 0xc000498498, 0x9, 0x9, 0xc005c964e0, 0xc006f02ca8, 0xc000487c68)
/opt/hostedtoolcache/go/1.16.10/x64/src/bufio/bufio.go:227 +0x222
io.ReadAtLeast(0x198bba0, 0xc0004e8d20, 0xc000498498, 0x9, 0x9, 0x9, 0x0, 0x6c4f18073adb01, 0xc006f02c90)
/opt/hostedtoolcache/go/1.16.10/x64/src/io/io.go:328 +0x87
io.ReadFull(...)
/opt/hostedtoolcache/go/1.16.10/x64/src/io/io.go:347
golang.org/x/net/http2.readFrameHeader(0xc000498498, 0x9, 0x9, 0x198bba0, 0xc0004e8d20, 0x0, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/frame.go:237 +0x89
golang.org/x/net/http2.(*Framer).ReadFrame(0xc000498460, 0xc002bb35c0, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/frame.go:492 +0xa5
golang.org/x/net/http2.(*clientConnReadLoop).run(0xc000487fa8, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/transport.go:1821 +0xd8
golang.org/x/net/http2.(*ClientConn).readLoop(0xc000481800)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/transport.go:1743 +0x6f
created by golang.org/x/net/http2.(*Transport).newClientConn
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/transport.go:695 +0x6c5
goroutine 4243 [sync.Cond.Wait, 6 minutes]:
sync.runtime_notifyListWait(0xc0065a7900, 0xc000000000)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0065a78f0)
/opt/hostedtoolcache/go/1.16.10/x64/src/sync/cond.go:56 +0x99
golang.org/x/net/http2.(*pipe).Read(0xc0065a78e8, 0xc0069d1c00, 0x200, 0x200, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/pipe.go:65 +0x97
golang.org/x/net/http2.transportResponseBody.Read(0xc0065a78c0, 0xc0069d1c00, 0x200, 0x200, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/transport.go:2110 +0xaf
encoding/json.(*Decoder).refill(0xc00a88c000, 0x300000000000, 0x7eff82fd91a0)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:165 +0xeb
encoding/json.(*Decoder).readValue(0xc00a88c000, 0x0, 0x0, 0x15ef480)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:140 +0x1ff
encoding/json.(*Decoder).Decode(0xc00a88c000, 0x16198a0, 0xc0043f5ea8, 0x203000, 0x203000)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:63 +0x7c
k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Read(0xc006af4690, 0xc003cc8000, 0x400, 0x400, 0x40, 0x38, 0x1673ec0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/framer/framer.go:152 +0x1a8
k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode(0xc006922eb0, 0x0, 0x1995440, 0xc000777640, 0xc000539dc8, 0x4672e5, 0xc00014ea80, 0x200000003, 0xc00014ea80)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/serializer/streaming/streaming.go:77 +0x89
k8s.io/client-go/rest/watch.(*Decoder).Decode(0xc0064a2560, 0x203000, 0x203000, 0x1, 0x0, 0xc000539e98, 0x4d53dc)
/home/runner/go/pkg/mod/k8s.io/[email protected]/rest/watch/decoder.go:49 +0x6e
k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc000777600)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/watch/streamwatcher.go:105 +0xe5
created by k8s.io/apimachinery/pkg/watch.NewStreamWatcher
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/watch/streamwatcher.go:76 +0xea
goroutine 4733 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc006f02ca0, 0x42f)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc006f02c90)
/opt/hostedtoolcache/go/1.16.10/x64/src/sync/cond.go:56 +0x99
golang.org/x/net/http2.(*pipe).Read(0xc006f02c88, 0xc00699e001, 0x3dff, 0x3dff, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/pipe.go:65 +0x97
golang.org/x/net/http2.transportResponseBody.Read(0xc006f02c60, 0xc00699e001, 0x3dff, 0x3dff, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/transport.go:2110 +0xaf
encoding/json.(*Decoder).refill(0xc006f03e40, 0xa, 0x9)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:165 +0xeb
encoding/json.(*Decoder).readValue(0xc006f03e40, 0x0, 0x0, 0x15ef480)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:140 +0x1ff
encoding/json.(*Decoder).Decode(0xc006f03e40, 0x16198a0, 0xc0040d3098, 0x203000, 0x203000)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:63 +0x7c
k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Read(0xc008ef7140, 0xc007df6a80, 0x4000, 0x4a80, 0x40, 0x38, 0x1673ec0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/framer/framer.go:152 +0x1a8
k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode(0xc002aa5900, 0x0, 0x1995440, 0xc002a9b080, 0x0, 0x0, 0x0, 0xc0004b3e64, 0x2)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/serializer/streaming/streaming.go:77 +0x89
k8s.io/client-go/rest/watch.(*Decoder).Decode(0xc00007dc80, 0xc0004b3e60, 0x0, 0x1, 0x1, 0x1, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/rest/watch/decoder.go:49 +0x6e
k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc002a9ad00)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/watch/streamwatcher.go:105 +0xe5
created by k8s.io/apimachinery/pkg/watch.NewStreamWatcher
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/watch/streamwatcher.go:76 +0xea
goroutine 4956 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc00634ab40, 0x26)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc00634ab30)
/opt/hostedtoolcache/go/1.16.10/x64/src/sync/cond.go:56 +0x99
golang.org/x/net/http2.(*pipe).Read(0xc00634ab28, 0xc0053c8001, 0x5ff, 0x5ff, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/pipe.go:65 +0x97
golang.org/x/net/http2.transportResponseBody.Read(0xc00634ab00, 0xc0053c8001, 0x5ff, 0x5ff, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/transport.go:2110 +0xaf
encoding/json.(*Decoder).refill(0xc004e85e40, 0xa, 0x9)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:165 +0xeb
encoding/json.(*Decoder).readValue(0xc004e85e40, 0x0, 0x0, 0x15ef480)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:140 +0x1ff
encoding/json.(*Decoder).Decode(0xc004e85e40, 0x16198a0, 0xc0039426d8, 0x203002, 0x203002)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:63 +0x7c
k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Read(0xc000cc5c20, 0xc004052800, 0x400, 0x400, 0x40, 0x38, 0x1673ec0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/framer/framer.go:152 +0x1a8
k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode(0xc003502780, 0x0, 0x1995440, 0xc00a6a0a40, 0x0, 0x0, 0x0, 0xc002b9de64, 0x2)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/serializer/streaming/streaming.go:77 +0x89
k8s.io/client-go/rest/watch.(*Decoder).Decode(0xc00007c840, 0xc002b9de60, 0x0, 0x1, 0x1, 0x1, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/rest/watch/decoder.go:49 +0x6e
k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc00a26a3c0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/watch/streamwatcher.go:105 +0xe5
created by k8s.io/apimachinery/pkg/watch.NewStreamWatcher
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/watch/streamwatcher.go:76 +0xea
goroutine 59 [select, 15 minutes]:
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch.func2(0xc0004981c0, 0xc000092300, 0xc001157d40, 0xc0062d1980)
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:373 +0x125
created by k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch
/home/runner/go/pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:367 +0x2ed
goroutine 5028 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0092e7e80, 0x52)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc0092e7e70)
/opt/hostedtoolcache/go/1.16.10/x64/src/sync/cond.go:56 +0x99
golang.org/x/net/http2.(*pipe).Read(0xc0092e7e68, 0xc0006de001, 0xdff, 0xdff, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/pipe.go:65 +0x97
golang.org/x/net/http2.transportResponseBody.Read(0xc0092e7e40, 0xc0006de001, 0xdff, 0xdff, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/transport.go:2110 +0xaf
encoding/json.(*Decoder).refill(0xc006d971e0, 0xa, 0x9)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:165 +0xeb
encoding/json.(*Decoder).readValue(0xc006d971e0, 0x0, 0x0, 0x15ef480)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:140 +0x1ff
encoding/json.(*Decoder).Decode(0xc006d971e0, 0x16198a0, 0xc003f72990, 0x203002, 0x203002)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:63 +0x7c
k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Read(0xc006e17bc0, 0xc003ce9300, 0x1000, 0x1300, 0x40, 0x38, 0x1673ec0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/framer/framer.go:152 +0x1a8
k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode(0xc006922910, 0x0, 0x1995440, 0xc00a26ab80, 0x0, 0x0, 0x0, 0xc000073e64, 0x2)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/serializer/streaming/streaming.go:77 +0x89
k8s.io/client-go/rest/watch.(*Decoder).Decode(0xc00658edc0, 0xc000073e60, 0x0, 0x1, 0x1, 0x1, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/rest/watch/decoder.go:49 +0x6e
k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc000776d40)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/watch/streamwatcher.go:105 +0xe5
created by k8s.io/apimachinery/pkg/watch.NewStreamWatcher
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/watch/streamwatcher.go:76 +0xea
goroutine 5211 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc00aaa14e0, 0x47)
/opt/hostedtoolcache/go/1.16.10/x64/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc00aaa14d0)
/opt/hostedtoolcache/go/1.16.10/x64/src/sync/cond.go:56 +0x99
golang.org/x/net/http2.(*pipe).Read(0xc00aaa14c8, 0xc006820001, 0x3fdff, 0x3fdff, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/pipe.go:65 +0x97
golang.org/x/net/http2.transportResponseBody.Read(0xc00aaa14a0, 0xc006820001, 0x3fdff, 0x3fdff, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/transport.go:2110 +0xaf
encoding/json.(*Decoder).refill(0xc00aaa1760, 0xa, 0x9)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:165 +0xeb
encoding/json.(*Decoder).readValue(0xc00aaa1760, 0x0, 0x0, 0x15ef480)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:140 +0x1ff
encoding/json.(*Decoder).Decode(0xc00aaa1760, 0x16198a0, 0xc0040d2cf0, 0x203000, 0x203000)
/opt/hostedtoolcache/go/1.16.10/x64/src/encoding/json/stream.go:63 +0x7c
k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Read(0xc002ba1110, 0xc006db0000, 0x40000, 0x50000, 0x40, 0x38, 0x1673ec0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/util/framer/framer.go:152 +0x1a8
k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode(0xc000495680, 0x0, 0x1995440, 0xc002a9afc0, 0x0, 0x0, 0x0, 0xc0020fbe64, 0x2)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/runtime/serializer/streaming/streaming.go:77 +0x89
k8s.io/client-go/rest/watch.(*Decoder).Decode(0xc0003acf00, 0xc0020fbe60, 0x0, 0x1, 0x1, 0x1, 0x0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/rest/watch/decoder.go:49 +0x6e
k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc0009637c0)
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/watch/streamwatcher.go:105 +0xe5
created by k8s.io/apimachinery/pkg/watch.NewStreamWatcher
/home/runner/go/pkg/mod/k8s.io/[email protected]/pkg/watch/streamwatcher.go:76 +0xea
There has not been any activity to this issue in the last 14 days. It will automatically be closed after 7 more days. Remove the stale
label to prevent this.
Thanks for the thorough report, and apologies for the late reaction. I'll investigate when I find the time. In the meantime, PRs are also welcome. 🙂
We are facing the same problem with the latest version