Benjamin Zaitlen

Results 198 comments of Benjamin Zaitlen

I can reproduce and I think I triaged the issue down to https://github.com/dask/distributed/pull/7564 . I don't have a lot of time to figure out what's going on . I'm not...

Would using `total_resources` or `available_resources` work for you ? I believe these are now in the [state_machine](https://github.com/dask/distributed/blob/main/distributed/worker_state_machine.py) of the worker ```python In [6]: client.run(lambda dask_worker: dask_worker.state.total_resources) Out[6]: {'tcp://127.0.0.1:57326': {'GPU': 2.0}}...

@lmeyerov thanks for moving the discussion here. If you have time, I think devs would also appreciate some motivating use cases or your thoughts on where mixing GPU/CPU workers is...

I don't have recommendations yet (first try attempting this). Naively I setup a cluster in the following manner: ### Scheduler dask-scheduler ### CPU Workers dask-worker tcp://...:8786 --nprocs 1 --resources CPU=1.0...

Hi folks, you might be interested in @madsbk recent work https://github.com/dask/distributed/pull/4869 allowing workers to have multiple executors

In rapidsai/cudf#3740 I linked to: https://docs.rapids.ai/api/dask-cuda/nightly/spilling.html

I believe we are still waiting on Tegra devices

Yes, is this something you'd be interested in coming back to ? This is on hold for a bit longer while we are exploring better spilling in general: https://github.com/rapidsai/cudf/pull/10746/