Server Request timeout response is delivered out of band

[johanandren]

Seen in the forums here: https://discuss.lightbend.com/t/akka-http-mapresponse-ignores-some-responses/1975/6

Intuitively request timeouts happens while in the routing tree, but actually that's not how they are implemented, which means that for example mapResponse does not also transform the request timeout response if the timeout hits.

Can we improve, and make mapResponse also map timeout responses out of the box, or something else that makes it more intuitive combining a route with the timeout?

If not, can we use documentation to make it more clear that it happens at a lower level?

[jrudolph]

Can we improve, and make mapResponse also map timeout responses out of the box, or something else that makes it more intuitive combining a route with the timeout?

The idea was that we have a low-level, low-effort safe-guard against long-running requests that will trigger even under load. The rationale was that any high-level (= routing) measures may under load try to do "too much" and delay the timeout. (A counter point would be that the way it is built, the timeout may have triggered and processing still cannot be cancelled, so even more work is done.)

[jrudolph]

Historically, there was a Timeout message in spray that your request handler actor could react to. If it didn't in a while, a TimeoutTimeout message was generated and a default timeout response was produced by the lower level.

[LMnet]

I'm the author of the original question on the forum and I want to add some thoughts here.

Initially, I thought that if I wrap all my routes with mapResponse it means that in this mapResponse I will be able to handle all responses. I found this behavior pretty intuitive. But in practice, some special responses will be ignored. This is the main WTF factor for me. This behavior is pretty unobvious and surprising.

Some vivid example - JSON API. It's a pretty common use-case when you want to wrap all your responses in the JSON, including all errors. Typical solution - wrap all your routes with the mapResponse and apply JSON wrapping there. But in reality it's not enough - you should add extra withRequestTimeoutResponse for the timeout errors. Also, is there any other extra responses like this which I should handle to create real JSON API?

Even if this behavior would be in the documentation for the mapResponse (and maybe some other directives), In my opinion, this will still remain unintuitive behavior.

At the current moment, I don't have any 100% better solution. Maybe we need some function like Route.seal, but for responses? Something like this:

mapResponse(...) {
  Route.responseSeal {
    // some routes
  }
}

In this code, I will have guarantees that all responses are sealed in my routes. And top-level mapResponse will be able to handle all routes in that case. Maybe Route.seal should do this?

[jrudolph]

One way we could deal with it would be to introduce a high-level timeout directive that would use the regular routing tree for providing timeouts. This would still not solve the original problem that the core layer needs to have a way of overriding any user processing in the case where the user processing just doesn't provide any response in time.

One way forward could be to:

• provide routing-level timeout directive that just uses the regular routing tree with all its features
• simplify the low-level timeout: log a warning (pointing to the high-level directive) and sent out a configurable (but not per-request-overridable) response

The advantage would be that mostly everything would be run through the routing tree with no surprises while we could still have a (longish) core-level timeout just as a fallback. Also, this would be an opportunity to remove the quite expensive timeout logic we currently have on the lower levels. Alternatively, we could just remove the low-level timeout and piggy-back on the idle-timeout (but then without giving any response).