client-go icon indicating copy to clipboard operation
client-go copied to clipboard

Is it possible to make the error retrieving resource lock for leader election Fatal?

Open smg247 opened this issue 1 year ago • 10 comments

We have recently uncovered an error in our boskos instance like the following:

E0710 21:02:45.211259 1 leaderelection.go:330] error retrieving resource lock ci/boskos-cleaner-leaderlock: leases.coordination.k8s.io "boskos-cleaner-leaderlock" is forbidden: User "system:serviceaccount:ci:boskos" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "ci"

This went unnoticed for quite some time as it is not a fatal error so the pod wasn't killed (and our alerting was never triggered). Is it possible to make the following errors Fatal?

  • https://github.com/kubernetes/client-go/blob/master/tools/leaderelection/leaderelection.go#L332
  • https://github.com/kubernetes/client-go/blob/master/tools/leaderelection/leaderelection.go#L336

smg247 avatar Jul 11 '23 18:07 smg247