Generic Cluster object for when you just have a scheduler address

[jsignell]

This came up in the dev meeting today, the idea is that there could be a Cluster object that behaves like the ProxyCluster in dask-ctl. This cluster would accept a scheduler address and not have any mechanisms for doing cluster management (like scale, or close).

One possible name for this would be RemoteCluster. The benefit of such a cluster is that it would allow us to do a clearer separation between cluster and client. So for instance the best practice for connecting to a generic external cluster would be:

current

from distributed import Client

client = Client("tcp://192.168.0.184:8788")

proposed

from distributed import RemoteCluster

client = RemoteCluster("tcp://192.168.0.184:8788").get_client()

The hope is that this syntax will make it easier for people to use different types of clusters and understand the client/cluster division of labor.

[mrocklin]

It's not yet clear to me what RemoteCluster would provide. Client's don't need a Cluster object to operate, they just need a scheduler address. If the desire is to support dask-ctl then maybe this feels a little bit like the tail wagging the dog.

[jacobtomlinson]

I think the only benefit here would be consistency. If folks are used to calling cluster.get_logs() when using LocalCluster they may benefit from the same API when accessing a remote cluster.

The motivation seems pretty similar to that in #6732 of meeting user expectations consistently.

Dask Control relies on ProxyCluster heavily because it makes an assumption that every cluster can be represented by an object that implements the Cluster interface. So if/when it comes time to upstream the lifecycle things from there we will need something like this, or to remove that assumption and special case when there is no cluster object representation.