dask-mpi

dask-mpi copied to clipboard

Implementing MPI communication protocol for Dask

5

I am currently experimenting with dask-mpi to replace point-to-point communication between schedulerworkers and workersworkers to use MPI. The goal is to exploit high-performance interconnects . I am aware of the...

aamirshafi

One worker does not shut down after code has finished running

7

**What happened**: Code is run with `mpirun` across two nodes each with 24 cores. Dask launches scheduler on one core and 46 workers as expected. Code executes as expected and...

kjgoodrick

Launching dask-mpi clusters

24

Is this supported? We have the ability (through Slurm) to `srun ... dask-mpi...` which will launch the cluster on a queue set aside for interactive work. From the command line...

rcthomas

I'm sitting with @zonca and he's asking how to start a Dask application from within an mpi4py application. I'll give a brief explanation, and then some code snippets. You can...

mrocklin

MPICluster

10

After the first implementation of the `initialize` function, making it possible to launch a Dask cluster from within the client script by using `mpi4py`, it seems like the natural next...

kmpaul