Guido Imperiale

Is it possible to write a unit test that deterministically reproduces the problem? As of #7062, the failure rate is 1/162.

@gjoseph92 any update on this?

> A `transitions` cycle holds control of the event loop until it's done. So I don't think it's possible for a worker to switch from running to non-running in between...

The good news: I pushed a deterministic reproducer. (work stealing is innocent). The bad news: The reproducer highlights an unrelated failure when queueing is active: #7204

> > @crusaderky your new test is passing for me; that's expected right? It seems like the `and lws.status == Status.running` check is solving the specific issue here, right? Yes,...

pseudocode: ```python if current managed_in_memory + self.comm_nbytes > threshold% * max_memory: stop scheduling further gather_dep's ``` or with better encapsulation: ```python if self.data.would_exceed_threshold(self.comm_nbytes): stop scheduling further gather_dep's ``` There are...

> 1. We could try to learn the size of transfers on the worker (creating something like a worker.py::TaskPrefix) > 2. We could ask the sender's side to cancel for...

> The worst case is when the data being sent is currently spilled to disk. `get_data` will then read it into memory (which stores a strong reference to it in...

> How would we get mypy to like these constants? As per everything mypy, we don't since mypy does not execute any code; it just searches for very specific explicit,...

int.to_bytes doesn't seem to have been implemented