Benjamin Zaitlen
Benjamin Zaitlen
We've ran into perm issues in the past, though MIG things might required something else. Possible solutions are documented here: https://github.com/gpuopenanalytics/pynvml#nvml-permissions
If we had the no-nanny option, at least for the CLI, users could then build workers manually > CUDA_VISIBLE_DEVICES=0 UCX_OPTION_FOO dask-cuda-worker ucx:// --no-nanny > CUDA_VISIBLE_DEVICES=1 UCX_OPTION_FOO dask-cuda-worker ucx:// --no-nanny
Thanks for following up @charlesbluca
I think it's a bit of work but definitely doable! 1. Before getting to cuda-workers, I _think_, we first need to resolve starting the scheduler in the cluster. Right now...
Dask on MPI systems are used and used quite a bit more than expected. You might be interested in the [dask-mpi](https://mpi.dask.org/en/latest/howitworks.html) project. @jacobtomlinson have you used `dask-mpi` to start a...
I think having some of these defaults makes sense . In dask core `--memory-limit` is set to `auto` by [default](https://distributed.dask.org/en/latest/worker.html) . ```python In [1]: import pynvml In [2]: pynvml.nvmlInit() In...
@ntabris are you able to move forward ?
Still an issue
The context statement is incorrectly stated: > async with await LocalCUDACluster Should be > async with LocalCUDACluster
Can you describe a bit more of the use case you have and why you might need both blazingsql and an additional async client ?