error decoding response body: deflate decompression error for garagehq.deuxfleurs.fr

[kevincox]

When trying to download content from garagehq.deuxfleurs.fr I get the following error when reading the request body:

Error downloading feed: error decoding response body: deflate decompression error

reqwest version: 0.11.9 features ["brotli", "deflate", "gzip", "json", "multipart", "rustls-tls-native-roots", "trust-dns"], hyper version: 0.14.16

Logs for request of https://garagehq.deuxfleurs.fr/rss.xml

[src/feeds.rs:191] &client = Client {
    accepts: Accepts {
        gzip: true,
        brotli: true,
        deflate: true,
    },
    proxies: [
        Proxy(
            System(
                {},
            ),
            None,
        ),
    ],
    referer: true,
    default_headers: {
        "accept": "*/*",
        "user-agent": "feedmail.org/0",
    },
    timeout: 600s,
}
[src/feeds.rs:195] &req = RequestBuilder {
    method: GET,
    url: Url {
        scheme: "https",
        cannot_be_a_base: false,
        username: "",
        password: None,
        host: Some(
            Domain(
                "garagehq.deuxfleurs.fr",
            ),
        ),
        port: None,
        path: "/rss.xml",
        query: None,
        fragment: None,
    },
    headers: {
        "accept": "application/atom+xml,application/rss+xml,text/html",
    },
}
[src/feeds.rs:204] res.headers() = {
    "content-type": "text/xml; charset=utf-8",
    "last-modified": "Mon, 07 Feb 2022 15:14:37 GMT",
    "accept-ranges": "bytes",
    "etag": "\"152840837968ef47c827a8954ca42d44\"",
    "date": "Tue, 08 Feb 2022 12:55:10 GMT",
}

Interestingly I can't see any Content-Encoding headers. Maybe they are stripped by hyper or reqwest before I can see them?

The website itself appears to support deflate, gzip and zstd compression which curl can read just fine.

curl -v --compressed -Haccept-encoding:deflate https://garagehq.deuxfleurs.fr/rss.xml
curl -v --compressed -Haccept-encoding:gzip https://garagehq.deuxfleurs.fr/rss.xml
curl -v --compressed -Haccept-encoding:zstd https://garagehq.deuxfleurs.fr/rss.xml

The server appears to have a server-side preference for zstd, then deflate. So it does make sense that the deflate content was being returned.

% curl -v --compressed -Haccept-encoding:br,gzip,deflate https://garagehq.deuxfleurs.fr/rss.xml
...
< content-encoding: deflate

Minimal Working Example

Create run the following program:

fn main() {
	let res = reqwest::blocking::get("https://garagehq.deuxfleurs.fr/rss.xml").unwrap();
	dbg!(&res);
	let body = res.text().unwrap();
	dbg!(&body);
}

It succeeds without the deflate feature and fails when deflate is added. Note that my full program is using the async interface.

[src/main.rs:3] &res = Response {
    url: Url {
        scheme: "https",
        cannot_be_a_base: false,
        username: "",
        password: None,
        host: Some(
            Domain(
                "garagehq.deuxfleurs.fr",
            ),
        ),
        port: None,
        path: "/rss.xml",
        query: None,
        fragment: None,
    },
    status: 200,
    headers: {
        "content-type": "text/xml; charset=utf-8",
        "last-modified": "Mon, 07 Feb 2022 15:14:37 GMT",
        "accept-ranges": "bytes",
        "etag": "\"152840837968ef47c827a8954ca42d44\"",
        "date": "Tue, 08 Feb 2022 13:15:17 GMT",
        "transfer-encoding": "chunked",
    },
}
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Custom { kind: Other, error: DecompressError(General { msg: None }) } }', src/main.rs:4:27
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[kevincox]

I found a list of websites supporting deflate and it looks like a significant number of them don't work with reqwest:

• https://www.roblox.com fails (server unspecified)
• https://www.Seznam.cz works
• https://Asana.com works
• https://Kp.ru/ fails (server: ycalb)
• https://36kr.com/ works
• https://profesionalci.rs/ works
• https://feralf.com/ works
• https://aptglobalmarine.com/ works
• https://Supermami.com.hk/ works

(omitted sites that didn't serve me deflate content)

[seanmonstar]

Would be curious what is different, and why the deflate library we're using is hitting an error only on some sites.