terraform-kubernetes-ebs-csi-driver
terraform-kubernetes-ebs-csi-driver copied to clipboard
Incorrect toleration used for ebs_csi_controller deployment
In the following line https://github.com/DrFaust92/terraform-kubernetes-ebs-csi-driver/blob/v3.10.0/controller.tf#L37 when var.csi_controller_tolerations is empty i.e. zero length, the deployment gets a broad toleration as below. This causes pods to be scheduled even on nodes with node.kubernetes.io/unschedulable:NoSchedule taint. Since this is not a daemonset what is the reason to have this broad toleration?
tolerations:
- operator: Exists
The toleration specified in that deployment, operator: Exists, is indeed the reason why the pod is being scheduled on a node with the node.kubernetes.io/unschedulable:NoSchedule taint.
Broad Tolerations: The operator: Exists in the tolerations section is a broad way to tolerate taints. It essentially means that the pod will tolerate any taint, regardless of its key, value, or effect.
Effect of Exists Operator: This kind of toleration doesn't specify a key or an effect. It simply tells Kubernetes that the pod can be scheduled onto any node, regardless of the taints that might be present on the node.
Implications: With such a toleration, Kubernetes's scheduler will consider the pod eligible for scheduling on any node, including those marked as unschedulable due to a NoSchedule taint.
This configuration is often used for system pods or in scenarios where you need certain workloads to run on every node regardless of their conditions (like DaemonSets). However, it's generally advisable to use more specific tolerations in typical application deployments to avoid unintended scheduling on nodes that might be unsuitable or reserved for specific purposes.
To restrict the pod from being scheduled on such nodes, you would need to replace the broad Exists toleration with more specific tolerations that match only the taints you want to tolerate.