Florian Jetter

The timeout errors in row three appear to throw a couple of `distributed.core.AsyncTaskGroupClosedError: Cannot schedule a new coroutine function as the group is already closed.` errors

@gjoseph92 can you be more specific about what kind of validations are lacking and what needs to be done?

I've talked about this in various other issues and called this a "circuit breaker" pattern. I think the "remove-worker" message (I would probably use a slightly different name) should then...

Caching the environment and letting the cache expire every day sounds good to me

> Could we have a nightly job that installs the env, runs the tests and only caches it when everything passes? Then all the PR jobs just use the most...

FYI the current algorithm already has a rough estimation for memory pressure implemented. It is not exclusively looking at CPU, see https://github.com/dask/distributed/blob/192a8bbe29b06db347da4bdd181383ebf952959e/distributed/scheduler.py#L7249-L7255 Of course, this logic may need some adjustment....

Not sure how best to add comments to the above table. I added another row with comments. | Events | Main | This PR | Comment | | --- |...

I'll have a closer look at the code and will provide more feedback about this proposal. I'm currently a bit skeptical about removing it since I _think_ we need it...

This is possibly closed by https://github.com/dask/distributed/pull/6847

My proposed course of action is - Introduce a config flag that toggles the profiler. - This config flag is on by default but will be disabled in our test...