akka-http icon indicating copy to clipboard operation
akka-http copied to clipboard

Route can't complete response until server request timeout if the client close the connection ungracefully

Open hellocome opened this issue 11 months ago • 1 comments

When the client close the connection ungracefully - for example send RST without sending ACK/FIN, RouteDirectives can't complete a slow response until hitting the server request timeout, is this by design or bug?

Set Server request timeout to 5 seconds

 val slowResponseTime = 3.seconds

 path("abc") {
        val timeoutResponse = {
          HttpResponse(
            StatusCodes.EnhanceYourCalm,
            entity = formatCurrentMilliseconds() + ": Unable to serve response within time limit")
        }

        withRequestTimeoutResponse(request => {
          System.out.println(formatCurrentMilliseconds() + ": Unable to serve response within time limit = " + count.get())
          timeoutResponse
        }) {
          val slowFuture = {
            val futureResponse: Future[HttpResponse] = Future {
              Thread.sleep(slowResponseTime.toMillis) 
              System.out.println(formatCurrentMilliseconds() + ": response to abc is created")
              HttpResponse(entity = HttpEntity(ContentTypes.`text/html(UTF-8)`, "response to abc,it is slow"))
            }

            futureResponse
          }
          val response = slowFuture // very slow
          complete(response)
        }
      }

we can see debug log like, I would expect complete should be invoked if there is an error in the flow, Unable to serve response within time limit should not be displayed in this case or it should not wait until server request timeout

[DEBUG] [12/23/2024 10:11:24.485] [akka-http-server-system-akka.actor.internal-dispatcher-6] [akka://akka-http-server-system/system/IO-TCP/selectors/$a/0] New connection accepted
20241223 10:11:24:623: Get Request = 0
[DEBUG] [12/23/2024 10:11:26.488] [akka-http-server-system-akka.actor.internal-dispatcher-6] [akka://akka-http-server-system/system/IO-TCP/selectors/$a/1] Closing connection due to IO error java.io.IOException: An existing connection was forcibly closed by the remote host
20241223 10:11:27:635: response to abc is created
20241223 10:11:29:636: Unable to serve response within time limit

Python script to simulate ungraceful shutdown

import socket
import struct
import time

def send_http_request():
    client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    linger_struct = struct.pack('ii', 1, 0)
    client_socket.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, linger_struct)
    client_socket.connect(("localhost", 8080))
    request = "GET /abc HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n"
    client_socket.send(request.encode())
    time.sleep(2)
    client_socket.close()
    if client_socket:
        client_socket.close()

if __name__ == "__main__":
    send_http_request()

hellocome avatar Dec 23 '24 10:12 hellocome

Sorry for very late response. I agree, that seems somewhat surprising. An even easier reproducer would be:

val log = LoggerFactory.getLogger(classOf[QuickstartApp.type])
    val futureBinding = Http().newServerAt("localhost", 8080).bindFlow(Flow[HttpRequest].mapAsync(1) { req =>
      log.info("Got request")
      Future.never
    }.watchTermination() { (nu, done) =>
      done.onComplete(t => log.info(t.toString))
      nu
    })

When hit with your reproducer python client this will lead to:

[info] [2025-03-03 13:50:01,291] [INFO] [com.example.QuickstartApp$] [HelloAkkaHttpServer-akka.actor.default-dispatcher-5] [] - Got request
[info] [2025-03-03 13:50:03,286] [INFO] [com.example.QuickstartApp$] [HelloAkkaHttpServer-akka.actor.default-dispatcher-5] [] - Failure(akka.stream.StreamTcpException: The connection closed with error: Connection reset)
[info] [2025-03-03 13:50:06,308] [INFO] [akka.actor.ActorSystemImpl] [HelloAkkaHttpServer-akka.actor.default-dispatcher-8] [akka.actor.ActorSystemImpl(HelloAkkaHttpServer)] - Request timeout encountered for request [GET /boom Empty]

So even if the connection reset was observed by the underlying Akka stream TCP infra, the request timeout in Akka HTTP hits.

johanandren avatar Mar 03 '25 12:03 johanandren