Florian Jetter

FYI tests break because the stealing now behaves differently and the tests reflect that (good!). just wanted to get this out early in case smbd has input already.

Change 2B: relax the sentinel values in `steal_time_ratio`. While it is ok do use a short path for no-dependency tasks to save compute cost, we should not disallow work stealing...

> Whoops! Well then it sounds like it must be a great idea :) I'm not entirely certain to be honest. This is one of the cases where I would...

> Ah, my thinking (or rather Gabe's thinking) was that you would keep the workers with dependencies in the set Right... that sounds straight forward. there is a saying in...

(Code still needs to be updated) I removed all the changes around decide_worker and solely focused on the blacklisting of `cost_multiplier > 100` and `duration < 0.005` tasks (which are...

> if there are any major concerns you have on this PR that weren't listed above then please let me know. Dropping the sentinels will emphasize the importance of proper...

I'm curious what other people think about the usefulness of such a thing (we started building tests with this in our infrastructure since this is a problem for us). If...

FWIW, I would be comfortable enough with calling C API directly for this to be merged. The API seems to be stable since py3.7 (https://docs.python.org/3/c-api/init.html#c.PyThreadState_SetAsyncExc) If we add more tests...

My final concern here is that we're not using the stdlib `ThreadPoolExecutor` interface any longer but require an extended interface which breaks some assumption about our compatibility, see https://github.com/dask/distributed/blob/96d4fd43682c379f9b39b6dc55f568872b766a47/distributed/worker.py#L3433-L3445 Discussion...

@oshadura The build is failing due to some linting issue, in particular a formatting issue since we're using black for automatic code formatting. I can recommend installing [pre-commit](https://pre-commit.com/) for this....