Unexpected 502 response from Microsoft-IIS/10.0

[t089]

We are observing an unexpected 502 responses when using async http client. Using curl on the same URL results in a successful request.

Example:

curl -vs https://www.eurohome.es/ical-rent/404.html > /dev/null
* Host www.eurohome.es:443 was resolved.
* IPv6: (none)
* IPv4: 81.88.57.88
*   Trying 81.88.57.88:443...
* Connected to www.eurohome.es (81.88.57.88) port 443
[... snip ...]
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://www.eurohome.es/ical-rent/404.html
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: www.eurohome.es]
* [HTTP/2] [1] [:path: /ical-rent/404.html]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]
> GET /ical-rent/404.html HTTP/2
> Host: www.eurohome.es
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/2 404 
< cache-control: private
< content-type: text/html; charset=utf-8
< server: Microsoft-IIS/10.0
< set-cookie: dadaproaffinity=7ff23aa7431c1694d43dffd9ed6cc9c9a2bc852f274fb026be889a5345c6d8d4;Path=/;Domain=www.eurohome.es
< x-powered-by: ASP.NET
< x-powered-by: ARR/3.0
< date: Thu, 05 Dec 2024 16:20:32 GMT
< content-length: 4894
< 
{ [4894 bytes data]
* Connection #0 to host www.eurohome.es left intact

But running the same request through AsyncHTTPClient gives as 502 - Bad Gateway response.

swift run downloader https://www.eurohome.es/ical-rent/404.html -v

HTTPClientResponse(version: HTTP/2.0, status: 502 Bad Gateway, headers: [("content-type", "text/html"), ("server", "Microsoft-IIS/10.0"), ("date", "Thu, 05 Dec 2024 16:22:13 GMT"), ("content-length", "1477")], body: /* snip */)

The request will succeed (ie with a 404 code) when forcing http1Only on the client. Note though, that curl is able to successfully use http2.

Reproducer code

import AsyncHTTPClient
import ArgumentParser
import NIOHTTP1
import NIOCore

@main
struct Downloader: AsyncParsableCommand {

    @Argument
    var url: String

    @Option(name: .customShort("X"))
    var method: String = "GET"

    @Flag(name: .short)
    var verbose: Bool = false

    @Flag
    var http1Only: Bool = false

    func run() async throws {
        var config = HTTPClient.Configuration()
        if http1Only {
            config.httpVersion = .http1Only
        }
        let client = HTTPClient(configuration: config)
        defer {
            Task { try? await client.shutdown() }
        }

        

        var request = HTTPClientRequest(url: self.url)
        request.method = HTTPMethod(rawValue: self.method)

        if verbose {
            print(request)
        }

        let response = try await client.execute(request, deadline: NIODeadline.now() + .seconds(30))

        if verbose {
            print(response)
        }

        for try await chunk in response.body {
            print(String(decoding: chunk.readableBytesView, as: UTF8.self), terminator: "")
        }
    }
}

[Lukasa]

So I haven't tried to run the reproducer but this is sounding a lot like an IIS bug. AHC will send a zero-length DATA frame with END_STREAM in order to terminate its response, and this can flush out bugs in some HTTP/2 implementations. A similar example is #602, or #573.

If you quickly modify a fork and try unconditionally setting Content-Length: 0 in the headers for your GET, does that fix the issue?

[t089]

Seems like you are right.

This "patch" does indeed fix the issue. I guess there is currently no API to cleanly manipulate a request in the channel pipeline.

diff --git a/Sources/AsyncHTTPClient/ConnectionPool/HTTP2/HTTP2ClientRequestHandler.swift b/Sources/AsyncHTTPClient/ConnectionPool/HTTP2/HTTP2ClientRequestHandler.swift
index 5e105c0..417c040 100644
--- a/Sources/AsyncHTTPClient/ConnectionPool/HTTP2/HTTP2ClientRequestHandler.swift
+++ b/Sources/AsyncHTTPClient/ConnectionPool/HTTP2/HTTP2ClientRequestHandler.swift
@@ -260,6 +260,9 @@ final class HTTP2ClientRequestHandler: ChannelDuplexHandler {
     }
 
     private func sendRequestHead(_ head: HTTPRequestHead, sendEnd: Bool, context: ChannelHandlerContext) {
+        var head = head
+        head.headers.add(name: "Content-Length", value: "0")
+
         if sendEnd {
             context.write(self.wrapOutboundOut(.head(head)), promise: nil)
             context.write(self.wrapOutboundOut(.end(nil)), promise: nil)

[Lukasa]

There is not. As discussed in #602, I'm wondering about whether we should enable a "quirks" mode that lets you ask us to do certain things we shouldn't have to do but that might make things work better, and this would be one of them. I'd accept a patch offering such config.