tensorlakeai
Timeout propagation UX
It would be hard for people to configure their function timeouts when i.e. their function calls another function which uses dynamic timeouts via progress updates.
Proposal: When a calling function is blocked on a function call/future then its timeout doesn't count down.
This will require an FE protocol extension because Executor doesn't know right now if the alloc function is blocked right now or not and that's it.
We probably don't need the protocol to be extended. We get the Application spec in the executor, if the executor sees a blocking call go through for a function, it can automatically stop the timeout of the caller function. wdyt?
Executor doesn't see "blocking calls". It only sees watchers that get created for blocking and non-blocking function calls. There's no guaranteed relation between watchers and what user code is doing at the moment, like if it's blocked on a function call or not. The only way how we can not extend the protocol is put extra assumptions on Executor side about watchers which needs extra analysis if this is the right thing to do now and in longer term.