dask-kubernetes Support for other protocols

Currently, the operator has a hardcoded TCP protocol https://github.com/dask/dask-kubernetes/blob/92714da5785709726f85c4c6ec92451f5c23ad04/dask_kubernetes/operator/controller/controller.py#L155-L159

Sep 16 '23 17:09 BitTheByte

This is a great point, if the scheduler uses ws or ucx the controller wont be able to talk to it. Ideally, all communication would happen over the HTTP API but that isn't enabled by default right now.

Perhaps an annotation on the DaskCluster is the right way to handle setting the protocol for now?

Sep 18 '23 13:09 jacobtomlinson

Looks like I didn't provide much information, this issue is about scheduler <=> worker communication, I'd like my cluster to communicate over ws rather than the hardcoded tcp protocol since you also mentioned it, the operator would also need this information to be able to contact the scheduler, I'm thinking of something like

apiVersion: kubernetes.dask.org/v1
kind: DaskCluster
metadata:
  name: dask-f3a0c12f
  namespace: default
spec:
 protocol: "ws"
 scheduler:
  ....

Sep 18 '23 14:09 BitTheByte

Ah yep I see what you mean. I agree that extending the CRD to include a protocol field makes sense. We will need to handle #753 in order to do this.

Sep 18 '23 14:09 jacobtomlinson