hanging go routines after metal-db restart

[mwennrich]

$ kubectl get pods -A|grep metal-d
metal-control-plane                            metal-db-0                                                2/2     Running     0          4h43m

$ kubectl get pods -A|grep metal-api
metal-control-plane                            metal-api-69c7c48557-6v4r8                                1/1     Running     0          4d12h
metal-control-plane                            metal-api-69c7c48557-8vscq                                1/1     Running     0          4d12h
metal-control-plane                            metal-api-69c7c48557-dws7h                                1/1     Running     0          4d14h
metal-control-plane                            metal-api-69c7c48557-khbd4                                1/1     Running     0          4h40m
metal-control-plane                            metal-api-69c7c48557-lrmwg                                1/1     Running     0          4h40m

[mwennrich]

goroutine profile: total 1553
669 @ 0x439ad6 0x432153 0x4655e9 0x4da5b2 0x4db91a 0x4db908 0x5c24e9 0x5d2e45 0x498a3a 0xd4f913 0xd4f8e4 0xd4ba17 0xd4ac25 0x46ad41
#	0x4655e8	internal/poll.runtime_pollWait+0x88					/usr/local/go/src/runtime/netpoll.go:234
#	0x4da5b1	internal/poll.(*pollDesc).wait+0x31					/usr/local/go/src/internal/poll/fd_poll_runtime.go:84
#	0x4db919	internal/poll.(*pollDesc).waitRead+0x259				/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
#	0x4db907	internal/poll.(*FD).Read+0x247						/usr/local/go/src/internal/poll/fd_unix.go:167
#	0x5c24e8	net.(*netFD).Read+0x28							/usr/local/go/src/net/fd_posix.go:56
#	0x5d2e44	net.(*conn).Read+0x44							/usr/local/go/src/net/net.go:183
#	0x498a39	io.ReadAtLeast+0x99							/usr/local/go/src/io/io.go:328
#	0xd4f912	io.ReadFull+0x52							/usr/local/go/src/io/io.go:347
#	0xd4f8e3	gopkg.in/rethinkdb/rethinkdb-go%2ev6.(*Connection).read+0x23		/go/pkg/mod/gopkg.in/rethinkdb/[email protected]/connection_helper.go:16
#	0xd4ba16	gopkg.in/rethinkdb/rethinkdb-go%2ev6.(*Connection).readResponse+0x56	/go/pkg/mod/gopkg.in/rethinkdb/[email protected]/connection.go:405
#	0xd4ac24	gopkg.in/rethinkdb/rethinkdb-go%2ev6.(*Connection).readSocket+0x24	/go/pkg/mod/gopkg.in/rethinkdb/[email protected]/connection.go:258

669 @ 0x439ad6 0x449872 0xd4ae33 0x46ad41
#	0xd4ae32	gopkg.in/rethinkdb/rethinkdb-go%2ev6.(*Connection).processResponses+0x192	/go/pkg/mod/gopkg.in/rethinkdb/[email protected]/connection.go:282

[Gerrit91]

I am unable to reproduce this. Could be the #271 helps in this scenario, but don't know.