GZip decompress response text automatically?

[drhumlen]

We've been struggling with some cryptic responses when trying to parse the json body response from some api we're using. Turned out it was because the response was gzipped. After some trial-and-error, this did the trick:

let! bodyByteStream = resp |> Response.toStreamAsync
use decompressionStream = new GZipStream(bodyByteStream, CompressionMode.Decompress)
let! decompressedBody = decompressionStream |> Stream.readUtf8StringAsync 48_000

The question is, should it be up to each individual FsHttp user to figure out and add decompression steps, or should it be incorporated into the library? Gzip is very common, so maybe there should be an easy way to add it?

I can try to make a PR for it if you can guide me in the general direction. But only if it's in the scope of what FsHttp is supposed to do?

[SchlenkR]

Thank you @drhumlen for the suggestion! This is an interesting idea.

My feeling is:

• We should do that automatically (maybe with an opt-out config switch) for response transformations like toString, toJson, toXml, etc. because it doesn't make sense to decode the compressed data into the target format.
• For functions like toStream, saveFile, toBytes, etc., decoding should not be done automatically. We could provide additional function counterparts for convenience like toDecodedStream, saveDecodedFile and toDecodedBytes (not sure of the naming; or we could switch the Response module to a type with static methods and overloads instead).
• We could maybe also provide a public functions like
  • Response.deflate: Response -> Stream
  • or Response.deflate: Response -> Response (I think I prefer that one)
• We should also take other common encodings in consideration for support, like deflate or Brotli.
• We should think about failure in case of an unsupported encoding when we change the semantics like described above.

[SchlenkR]

I changed my opinion which I stated earlier. Since compression is specified on http protocol level, specification of decompression should be the same for all response content functions. Therefore, it's a configuration aspect of the initial request or the returned response.