Matthew Rocklin

In practice I doubt that the scheduler will be expensive enough that system administrators will care. They all ask about this, but I don't think that it will be important...

We need to make an object that follows this interface: https://github.com/dask/distributed/blob/e7a2e6d41e0b719866769713d8f41cb5fcfbf6e8/distributed/deploy/spec.py#L22-L30 The simplest example today is probably SSHCluster: https://github.com/dask/distributed/blob/master/distributed/deploy/ssh.py But rather than start things with SSH, it would presumably submit...

There are a few here: https://blog.dask.org/2019/08/28/dask-on-summit On Wed, Oct 16, 2019 at 10:22 PM Loïc Estève wrote: > I wish there were some list of cluster configuration quirks (that are...

+1 from me if it's doable. This would be great. On Tue, May 19, 2020 at 2:08 AM Loïc Estève wrote: > from dask_jobqueue import SLURMCluster cluster = SLURMCluster(cores=1, memory='1GB')...

I did. Things worked, but weren't yet any faster. The Dask + UCX team within RAPIDS (which @quasiben leads) is working on profiling and performance now, so hopefully we'll see...

Are there any job schedulers that we might support in the future that would not have these keyword arguments? On Tue, Nov 20, 2018 at 11:27 AM Guillaume Eynard-Bontemps <...

If you think that daskec2lite is a good replacement for dask-ec2 I recommend making it more visible first by raising an issue that asks people to investigate it, and then...

In principle this seems fine to me. Thank you for paying attention to the test suite. I agree that it could be better unified.

Dask uses disk to spill excess data. I would expect it to be at least as large as RAM, if not larger. I don't know what level is correct, but...

I would also like to see this. I wonder if @jbcrail has time to take a look at this.