Florian Jetter

Adjust threshold for bad worker on worker table based on worker-ttl

2

We're currently marking workers as red on the worker table page if they don't heartbeat within 60s. The 60s is quite arbitrary and particularly if users expect heartbeats not to...

Use TaskSpec in local dask execution

1

This builds on https://github.com/dask/dask/pull/11377 and includes https://github.com/dask/dask/pull/11347 This effectively causes all dask/dask schedulers to convert to the new task spec and use it as an execution backend. Todos - Haven't...

Consider using object pools for internal state keeping

3

Dask is creating many small objects and not just for very large graphs but this is pretty much a built in thing. This is one of not the most prominent...

Add retry to shuffle broadcast

3

One of our P2P stress tests is failing pretty regularly `distributed.tests.test_stress.test_close_connections`  This failure is somewhat expected because the broadcast is connecting to all workers and the connection attempts may...

Remove idle and saturated sets from scheduler

1

This is an attempt at removing the `idle`/`saturated`/`idle_task_count` containers on the scheduler. - They are expensive to maintain. They inherently rely on occupancy which is a quantity that cannot be...