gcp-compute-persistent-disk-csi-driver icon indicating copy to clipboard operation
gcp-compute-persistent-disk-csi-driver copied to clipboard

Node DaemonSet in Kubernetes Deployment Should Not Run On Master

Open davidz627 opened this issue 7 years ago • 6 comments

Currently the Daemonset constantly attempts to create the pod on master and fails because of resource limits. We should stop scheduling on master so that it does not continually loop attempted creation.

This is high priority as we have seen the daemonset pod kick off the api server from the master putting the whole cluster in a bad state.

davidz627 avatar Jul 02 '18 22:07 davidz627

Looking at my master node,

Taints:             <none>
Unschedulable:      true

My master daemonset pod:

Tolerations:     node.kubernetes.io/disk-pressure:NoSchedule
                 node.kubernetes.io/memory-pressure:NoSchedule
                 node.kubernetes.io/not-ready:NoExecute
                 node.kubernetes.io/unreachable:NoExecute

@janetkuo, do you know how to prevent daemonset pods from being scheduled on the master?

msau42 avatar Jul 02 '18 23:07 msau42

(this is gce, not gke)

msau42 avatar Jul 02 '18 23:07 msau42

From https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/:

The unschedulable field of a node is not respected by the DaemonSet controller.

Looks like taints are respected though, but my thought is tainting the master node could have unwanted side-effects

davidz627 avatar Jul 03 '18 00:07 davidz627

I don't know if there are any such addons that want to run on all nodes, including the master.

msau42 avatar Jul 03 '18 00:07 msau42

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot avatar Apr 26 '19 00:04 fejta-bot

/remove-lifecycle stale /lifecycle frozen

msau42 avatar Apr 30 '19 01:04 msau42