cloudflared icon indicating copy to clipboard operation
cloudflared copied to clipboard

fix s3 streaming uploads broken by go/net bump

Open sh4dowb opened this issue 5 months ago • 1 comments

fixes #1511

cloudflared sets NoBody for non chunked and zero content length requests as seen below:

        // Go's client defaults to chunked encoding after a 200ms delay if the following cases are true:
        //   * the request body blocks
        //   * the content length is not set (or set to -1)
        //   * the method doesn't usually have a body (GET, HEAD, DELETE, ...)
        //   * there is no transfer-encoding=chunked already set.
        // So, if transfer cannot be chunked and content length is 0, we dont set a request body.
        if !isWebsocket && !isTransferEncodingChunked(req) && req.ContentLength == 0 {
                req.Body = http.NoBody
        }

However, when content-length is not set, it defaults to 0, and AWS S3 uses content-encoding: aws-chunked instead of transfer-encoding, causing cloudflared to set NoBody. With the go/net module 0.26.0->0.40.0 bump, it explicitly sets content-length: 0 if request is NoBody here, even when original request does not have a content-length header. This causes S3-compatible storages, at least minio, and potentially other applications to fail, because the uploaded body size does not match the header.

The fix simply searches the string chunked in content-encoding alongside transfer-encoding, in order to skip setting NoBody.

sh4dowb avatar Aug 02 '25 00:08 sh4dowb

Hey, I had the same issue, thanks @sh4dowb!

eliassoares avatar Sep 14 '25 02:09 eliassoares