frugal
frugal copied to clipboard
Improve Timeout Information For Server Time Budgeting
Today, a server can check the Timeout()
on the FContext
it receives in order to estimate how much time is left to serve the request. However, this can be a significant overestimate due to:
- Network latency
- Time spent sitting in the server's request buffer
It would be great if Frugal could add (1) a best-effort wall time when the client request was made, (2) the amount of time a message spent in the queue, or both. This would help Frugal servers make better estimates of the true client time budget.
https://jira.atl.workiva.net/browse/MSG-448
It seems like the two options here are:
- Have clients put a deadline in the FContext before they make a request (either in addition to or in replace of a relative timeout). One flaw here is clock drift from browser clients could be excessive
- Have the server deserialize the FContext, put a deadline in from the relative timeout, then reserialize the FContext (ignoring network latency, but would catch time in the queue).
This is something we just ran into too