content-encoding response header gets stripped out

[carloseltuerto]

Once a filter Decompressor has done its work, it removes from the final response header the content-encoder key. It is true that corresponding Envoy Meta headers are returned on the onTrailers callback. Here is an example with Brotli:

'x-envoy-decompressor-gzip-compressed-bytes', '47'
'x-envoy-decompressor-gzip-uncompressed-bytes', '43'

Unfortunately, this information is not enough to rebuild the original content-encoding. Is says "gzip", and this is "Brotli" in reality.

This behaviour is documented here: https://github.com/envoyproxy/envoy/blob/main/docs/root/configuration/http/http_filters/decompressor_filter.rst

Those are the logs exposing the behaviour:

[2022-03-06 15:20:48.535][2941411][debug][filter] [external/envoy/source/extensions/filters/http/decompressor/decompressor_filter.cc:118] [C12345][S622517732233522859] DecompressorFilter::encodeHeaders: ':status', '200'
'content-encoding', 'br'
'x-envoy-attempt-count', '1'
'x-envoy-upstream-service-time', '144'

[2022-03-06 15:20:48.536][2941411][trace][filter] [external/envoy/source/extensions/filters/http/decompressor/decompressor_filter.h:183] [C12345][S622517732233522859] do decompress response: ':status', '200'
'x-envoy-attempt-count', '1'
'x-envoy-upstream-service-time', '144'

...

[2022-03-06 15:20:48.536][2941411][debug][http] [library/common/http/client.cc:68] [S0] dispatching to platform response headers for stream (end_stream=false):
':status', '200'
'x-envoy-attempt-count', '1'
'x-envoy-upstream-service-time', '144'
'date', 'Sun, 06 Mar 2022 15:20:48 GMT'

Cronet expects that header to be present.

cc: @alyssawilk, @goaway

[alyssawilk]

Wait, just to be clear, in Chrome they undo compression but leave the original header? If so I'm inclined to think that's broken behavior, but I'd support Envoy cloning it into x-envoy-downstream-content-encoding and cronet moving that back to content-encoding, or some such.

[carloseltuerto]

I do understand why this is wrong - you can't tell which decompression scheme(s) have been applied.

But this is how Cronet works - this is the original test where it is ensured that the content-encoder header does not get stripped:

https://source.chromium.org/chromium/chromium/src/+/main:components/cronet/android/test/javatests/src/org/chromium/net/BrotliTest.java;l=104

My guess is there was a provision to tell which decompression scheme(s) have been applied by envoyMobile. However that mechanism needs help, as far as I can tell. This is what we get today from EnvoyMobile:

'x-envoy-decompressor-gzip-compressed-bytes', '47'
'x-envoy-decompressor-gzip-uncompressed-bytes', '43'

It does not say that Brotli was used to decompress.

[alyssawilk]

yeah so I think in pure config we can add an envoy header modifier which basically says "copy original content encoding into x-envoy-original-content-encoding header". I think that's cleaner than the proxy having config knobs to preserve, especially as we'd have to do it in every filter which tweaked content encoding.

[carloseltuerto]

Yes - that's a Cronet "specificity" - Cronvoy will retranslate back. Thanks.

[goaway]

Just to chime in, I like @alyssawilk's suggestion here.