Caddy seems to prefer Accept-Encodings left-to-right (prefers gzip over zstd)

[da2x]

Hi, I’ve been crawling the public internet to determine support for the Zstandard compression method. Caddy showed up in my crawls, but I noticed a potential issue:

“Accept-Encoding: deflate, gzip, br, zstd” (which is what Chrome and Firefox will begin sending shortly) always returns “Content-Encoding: gzip” in Caddy. However, if I move “zstd” to the beginning of the string, Caddy will return Zstandard encoded responses instead. For example, “Accept-Encoding: zstd, deflate, gzip, br”.

The order of a comma-separated list is not significant in HTTP. Changing the order should not affect the response. The server is always expected to pick the best encoding the client accepts. Generally, this is the smallest, fastest, and cheapest encoding.

(Suggestion) Ideally for static files, the first request would generate and cache a response with one supported encoding, repeat with a different encoding for the second request, and then any subsequent requests should receive the smaller of the two.

[WeidiDeng]

Caddy will pick whichever encoding client and server both prefer. There is no way to know which is better. In your example, caddy can be configured to prefer zstd. Maybe this behavior will be changed later when zstd is more widespread.

Generating compressed cache is outside of Caddy scope now, it's very complicated (permission, file changes, load etc.). Caddy can be configured to serve precompressed files. Caddy supports br when serving precompressed files when normal couldn't.

[francislavoie]

See https://github.com/caddyserver/caddy/pull/4045 and https://github.com/caddyserver/caddy/issues/2665 for background on the current behaviour.

The logic is here:

https://github.com/caddyserver/caddy/blob/a9768d2fdefeae8050f1d328e7133e312acd253f/modules/caddyhttp/encode/encode.go#L340-L414

[mholt]

I'm open to improving our content negotiation, but I agree with Weidi that keeping the state and caching responses and then comparing which is better, etc, is probably too complex, at least for something built-in. If this is that important to you (saving every byte possible) I'd recommend either precompressing the content (and choosing the best encoding format for each file you compress) which Caddy will serve out-of-the-box, or writing a plugin to do that really complicated optimization logic.

(If it turns out to be easy and simple, we can probably incorporate it. But I am skeptical...)

[bannmann]

In your example, caddy can be configured to prefer zstd. Maybe this behavior will be changed later when zstd is more widespread.

Well, I did not manage to do that! Configured with encode zstd gzip, Caddy 2.8.4 always seems to pick gzip when it's present in the client's Accept-Encoding header (e.g. accept-encoding: gzip, zstd).

However, the Caddy docs claim that

if the client has no strong preference (q-factor), then the first supported encoding is used.

What does "first supported encoding" refer to?

• the ordering in the request header
  • This is supposed to be irrelevant as @da2x writes
• the ordering in the config file
  • In this case, either the docs are wrong or Caddy has a bug here.

Can somebody reproduce or refute this? Maybe I messed up my tests. But if I didn't, Caddy users would never benefit from ZStandard support in Firefox, Chrome and Edge, as all these browsers also include gzip in their Accept-Encoding header.

[mholt]

@bannmann What is the client's exact Accept-Encoding header in your case?

[bannmann]

@mholt:

curl --header 'accept-encoding: zstd, gzip' -v MYURL > /dev/null
(...)
< content-encoding: gzip

I also just double-checked I have configured encode zstd gzip.

[bannmann]

More variations:

curl --header 'accept-encoding: gzip, zstd' -v MYURL > /dev/null
(...)
< content-encoding: gzip

curl --header 'accept-encoding: zstd' -v MYURL > /dev/null
(...)
< content-encoding: zstd

curl -v MYURL > /dev/null
(...)
(no content encoding header)

[yaakovfeldman]

I also find that gzip is used instead of zstd. My firefox sends Accept-Encoding | gzip, deflate, br, zstd and I get back gzip. I am using the encode directive with no params which the docs say is equivalent to encode zstd gzip. Perhaps this should be reopened? Is anyone getting zstd?

[tesinormed]

Here's a workaround:

@accepts-zstd header Accept-Encoding *zstd*
request_header @accepts-zstd Accept-Encoding zstd

[malberts]

I needed to normalise that header for caching purposes, which has the side-effect of forcing priority:

map {header.Accept-Encoding} {encoding} {
	~zstd "zstd"
	~br "br"
	default "gzip"
}
request_header Accept-Encoding {encoding}

[Forza-tng]

Here's a workaround:

@accepts-zstd header Accept-Encoding *zstd*
request_header @accepts-zstd Accept-Encoding zstd

Thanks! I expanded on this in order to determine multiple or different fallbacks. for fome reason I had to force the encoding inside each handle, rather than changing the request header, otherwise i got zstd encoded results instead of br. I chose br (github.com/dunglas/caddy-cbrotli) because it has a better compression ratio for the text, js, css and html files I have.

    #encode zstd br gzip
    route {
        @accepts-br header Accept-Encoding *br*
        @accepts-zstd header Accept-Encoding *zstd*
        handle @accepts-br {
            encode br
        }
        handle @accepts-zstd {
            encode zstd
        }
        handle {
            encode gzip
        }
    }