dask-kubernetes icon indicating copy to clipboard operation
dask-kubernetes copied to clipboard

Scheduler uses the same container template & name as the workers

Open pvanderlinden opened this issue 3 years ago • 2 comments

The scheduler will take the same name as a worker (so you don't know which pod is the scheduler by looking at the name) It also uses the same resource requests (which it doesn't actually need).

pvanderlinden avatar Jun 15 '21 10:06 pvanderlinden

@jacobtomlinson I have a similar problem using KubeCluster as I would like to deploy the scheduler pod away from the "hard-working" nodes as it's a central point of failure. it would be easier to let us have two separate pod_spec for worker and scheduler

BitTheByte avatar Apr 21 '22 07:04 BitTheByte

Having an optional scheduler pod spec sounds reasonable. Is this something you would be interested in contributing?

With the new operator on the horizon I am not doing much development on KubeCluster as I hope it will eventually be superseded, but happy to review contributions.

jacobtomlinson avatar Apr 25 '22 10:04 jacobtomlinson

The classic KubeCluster was removed in https://github.com/dask/dask-kubernetes/pull/890. All users will need to migrate to the Dask Operator. Closing.

jacobtomlinson avatar Apr 30 '24 15:04 jacobtomlinson