Benjamin Zaitlen

I believe what MP does is serialize the file, spawns a new process, deserializes the file, then executes the target function. I heard some rumors about how cloudpickle could be...

FWIW, the user survey from last year shows that the 2nd (1st is ssh) most popular way of launching dask is with HPC tooling: - https://blog.dask.org/images/2021_survey/2021_survey_31_0.png - https://blog.dask.org/2021/09/15/user-survey @jrbourbeau just...

I looked at this and there is a race condition but I am not sure that we can easily resolve inside of dask. The race condition being that as we...

add to allowlist

@idorrington92 the comment allows us to run your PR against a GPU CI system: https://docs.dask.org/en/stable/develop.html#gpu-ci Distributed has been flaky in the past and recently there has been a concerted effort...

I think caching seems like a good idea to speed up things. We could also build a nightly docker image of dependencies and run test in the image. This give...

@charlesbluca do you have time to try a test PR which implements the caching ?

Sorry, i took off for a bit. I was thinking about single app-wide tokens. Token generation should be handled by some other auth and be responsible for correctly generating the...

Can you elaborate on how this error came up ? Or better yet, build a minimal reproducer ?

> I'd have to figure out how to do all the CUDA context connecting, but this doesn't seem that bad for a proof of concept at least. I don't have...