Transparent (content-encoding) compression

[jimdigriz]

An initial stab at merging my wrapper into hackney its-self which addresses issue https://github.com/benoitc/hackney/issues/155

To use this, just pass compress as an option to hackney:{request,get,...}.

[seriyps]

Tests a failing because of a dialyzer warning:

_build/default/lib/hackney/src/hackney_http.erl
 111: Record construction #hparser{type::'auto',max_line_length::4096,max_empty_lines::10,empty_lines::0,state::'on_first_line',buffer::<<>>,method::<<>>,partial_headers::[],te::<<>>,ce::<<>>,connection::<<>>,ctype::<<>>,location::<<>>,body_state::'waiting',encoding::'undefined'} violates the declared type of field encoding::{atom(),_}
 523: Function parse_options/2 will never be called

I would suggest you to change type of encoding field to {atom(), any()} | undefined.

[fenollp]

Shouldn't the new compress option be inferred from the accept header instead of needing to be explicitly specified? Was it not done this way for backwards compatibility reasons?

[jimdigriz]

I used compress to avoid having to create a API call so then the application can tell what to put in its headers (today we support deflate/gzip simply as it brings in no dependencies). I suspect to a developer most of the time they think more "compress where possible" rather than "please only use deflate".

However, this is how I think of using hackney, if I need something else, I just hack the source and submit a PR, others I am willing to accept might prefer something else :)

[fenollp]

Works great for me. Thanks :)

[jimdigriz]

Finally found the time to improve the compress flag support; I moved the functionality from hackney_http into hackney_response which seems more appropriate as Content-Encoding is application layer feature rather than part of the transport.

I chose to not add guards to skip initialising zlib for clen=0/bad_int and HEAD requests as the user chooses those values and could also choose to not use compress too. This was just my preference, I am not wedded to it and happy to change that behaviour.

I went to try adding unit tests, but looking around it seems to be non-trivial to add gzip/deflate support to gunicorn so got stopped in my tracks. Any suggestions on what to do here?

[benoitc]

@jimdigriz i added the tests using the new test platform. will land with the next RC.

[jadeallenx]

@benoitc Any update on this?