Florian Jetter

There are three tasks that can be worked on separately - The scheduler tells all workers that a worker left. The worker then adjusts internal state, e.g. who_has dicts -...

cc @graingert

I can confirm that @gjoseph92 s analysis is correct and this is an ordering issue. I see two options 1. Scatter uses the batched comm as well and follows ordering....

Decoupling scatter_to_workers from update_data as suggested above in 1) indeed resolves the race that triggers the cancelled error but it introduces a similar race condition on worker side resulting in...

> We could make a new fsspec-all metapackage depending on exact versions of fsspec/s3fs/gcsfs (maybe adlfs too) and change the envs to depend on that instead. I'm more concerned about...

I would like input from @jrbourbeau but IIUC the current implementation doesn't allow us to distinguish much here. If this toggle is on, a `map_partitions` is scheduled that will cast...

Yes, the `date` type should be better handled this way. I was thinking mostly about the lists and more complex things.

Looks like there are a bunch of genuine typing issues since I set the dtype on the created series explicitly. @phofl if you want to, you can pick this up...

The dask tokenizer is exposed to a more or less similar problem. We recently tried to solve this for tokenize but couldn't come up with a stable version that is...

Thanks for opening this issue. First of all, there is actually a minor bug in your code since you should be using `dask.compute` inside of the delayed driver function (`do_math`)...