kube-rbac-proxy icon indicating copy to clipboard operation
kube-rbac-proxy copied to clipboard

Add livenessProbe support for kube-rbac-proxy

Open jessehu opened this issue 1 year ago • 2 comments

When using kube-rbac-proxy v0.14.1, sometimes we found kube-rbac-proxy container stuck in TLS handshake error and can not recover automatically until it's restarted manually.

I0621 17:39:33.819787    1876 log.go:198] http: TLS handshake error from 10.255.9.20:55542: write tcp 10.255.9.20:9100->10.255.9.20:55542: write: broken pipe
I0621 17:40:40.388479    1876 log.go:198] http: TLS handshake error from 10.255.9.26:1531: write tcp 10.255.9.20:9100->10.255.9.26:1531: write: broken pipe
I0621 17:44:32.288256    1876 log.go:198] http: TLS handshake error from 10.255.9.26:30302: write tcp 10.255.9.20:9100->10.255.9.26:30302: write: broken pipe

When used with prometheus node-exporter, adding a livenessProbe can automatically restart kube-rbac-proxy container, e.g.:

        - image: myregistry.io/kube-rbac-proxy:v0.14.1-with-curl
          livenessProbe:
            exec:
              command:
                - /bin/sh
                - -c
                - "curl -sSL -ik -H \"Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)\" https://${IP}:9100/metrics | grep -e 'HTTP/2 200'"
            initialDelaySeconds: 30
            failureThreshold: 3
            periodSeconds: 30
            successThreshold: 1
            timeoutSeconds: 3

jessehu avatar Jun 21 '23 09:06 jessehu