"invalid header value" error on some websites

[Shnatsel]

On some websites, e.g. http://ceip.org, surf fails with the following error:

invalid header value

Firefox, curl and ureq (a blocking Rust client) work fine.

3888 websites out of the top million from Feb 3 Tranco list are affected.

Tested using this code. Test tool output from all affected websites: surf-invalid-header-value.tar.gz

I've only tested the async-h1 backend; I don't know if the other backends are affected.

[Fishrock123]

All of these work (see https://crates.io/crates/longboard, which uses Surf):

longboard GET http://ceip.org
longboard GET https://ceip.org
longboard GET https://carnegieendowment.org/

I'm unable to reproduce.

[Shnatsel]

I can still reproduce with this code.

My code follows redirections and dumps headers to stdout. Could longboard be unaffected because it doesn't ever look at the headers?

[06chaynes]

I am also seeing a malformed header value in a response, specifically the etag header. I was using http://slowglowingyoungkiss.neverssl.com/online as the endpoint to test against.

Here's the relevant portion of the response from curl, where etag is properly formed:

etag: "1727356907f2ef9c00e6d0d3c44fd95b"

This is the header when the response is retrieved with surf (default features):

etag: W/"1727356907f2ef9c00e6d0d3c44fd95b"

On each test I get this W/ before the value, so far I've only seen this with the etag header.