Middleware: compression

[yoshuawuyts]

Using https://docs.rs/async-compression/0.1.0-alpha.1/async_compression/ we should write a middleware that sends Accept-Encoding parameters to the server, and decompresses the response.

Examples

• https://github.com/rustasync/tide/blob/master/tide-compression/src/lib.rs

[Nehliin]

Hi, I would like to help with this as a learning experience if that's okay? I looked at the example code from the Tide implementation and will plan to use that as inspiration. The end goal would be something like:

 let string = surf::get(uri)
    .middleware(ResponseCompression::default())
    .recv_string().await?;

I used the name ResponseCompression (Happy to change it if you want) since I assumed this would not handle compression of the body for the outgoing request, just the response back and set the Accept-Encoding headers for the outgoing request. The middleware would set the header with br, gzip, zstd and deflate and then decode the server response.

To test this I thought I could implement a stub HttpClient that could just respond with compressed data. Have I understood the issue correctly? Or did you mean that the middleware also should handle compression for outgoing requests and then set the Content-Encoding header for the request?

[yoshuawuyts]

@Nehliin That would be fantastic, thanks! I feel like you're mostly on the right path here for sure!

Or did you mean that the middleware also should handle compression for outgoing requests and then set the Content-Encoding header for the request?

This isn't quite possible unfortunately because by default we can't know what the client accepts. HTTP/2 applies some compression by default to circumvent this, so I wouldn't worry too much about it. Decompressing server responses is by far the right priority here!

[Nehliin]

Okay great! I'll start this week

[Nehliin]

I didn't quite have as much time this week as I wished but I thought I might give some status update at least. I have implemented a first version of the middleware but polish is still needed before I'll have a PR up. I have also implemented the StubClient which tests the decompression and that the right headers are set. I will write some more tests and clean the code up a bit afterwards. I sure have a lot to learn working with streams so I would love to get some feedback to make it more idiomatic when I have something to show you.

[yoshuawuyts]

@Nehliin yay, thanks for the update! -- and yeah for sure, happy to give feedback!

[Nehliin]

I changed my mind and thought I'd put up a WIP PR instead of trying to fix everything in isolation, the earlier the feedback the better!