kubermatic
Validating webhook for user cluster tolerations overrides
Description of the feature you would like to add / User story
As a KKP user - user-cluster owner, I would like to get immediate feedback when specifying incorrect tolerations for Cluster.spec.componentsOverride.userClusterController.tolerations.
Solution details
- kube-apiserver has the validating logic, we could embed that to kubermatic validating webhooks
Alternative approaches
n/a
Use cases
When adding broken tolerations to a Deployment, the kube-apiserver rejects the call with explanatory error message. Kubermatic Cluster CRD would allow it and then add warning event to the Cluster because the Deployment spec would get rejected
Warning ReconcilingError 12m (x6 over 27m) kkp-kubernetes-controller failed to reconcile cluster: failed to ensure Deployment cluster-h4hf7gb22w/usercluster-controller: failed to update object *v1.Deployment "cluster-h4hf7gb22w/usercluster-controller": Deployment.apps "usercluster-controller" is invalid: spec.template.spec.tolerations[0].operator: Invalid value: core.Toleration{Key:"key", Operator:"Exists", Value:"val", Effect:"NoExecute", TolerationSeconds:(*int64)(nil)}: value must be empty when `operator` is 'Exists'
Additional information
I want to work on this issue @wozniakjan , will you help me solving it?
hey @shrinu-kushagra, the best place would likely be cluster validation webhook https://github.com/kubermatic/kubermatic/blob/f0cb4e1a13576a1b32bdb829ccf2fe9323447dad/pkg/webhook/cluster/validation/validation.go#L69 https://github.com/kubermatic/kubermatic/blob/f0cb4e1a13576a1b32bdb829ccf2fe9323447dad/pkg/webhook/cluster/validation/validation.go#L105
and ideally implement similar validation logic for tolerations as the upstream Kubernetes uses for validating pod tolerations
