cli icon indicating copy to clipboard operation
cli copied to clipboard

πŸ› Bug Report: http --download misinterprets Content-Length when Content-Encoding: gzip is set

Open kohnalex opened this issue 5 months ago β€’ 0 comments

πŸ› Bug Report: http --download misinterprets Content-Length when Content-Encoding: gzip is set

Summary

When using httpie with --download to retrieve a file with Content-Encoding: gzip, the tool reports an "Incomplete download" even though the Content-Length is correctly set to the size of the compressed payload.

This behavior contradicts the HTTP/1.1 spec, which states that Content-Length must reflect the encoded (compressed) size when Content-Encoding is used.

Steps to Reproduce

  1. Serve a GZIP-compressed JSON file with these headers:
Content-Type: application/json
Content-Encoding: gzip
Content-Disposition: attachment; filename="data.json.gz"
Content-Length: 5084527  # actual size of compressed content
  1. Use HTTPie to download the file:
http --download GET http://localhost:3000/download
  1. Observe this error:
Incomplete download: size=5084527; downloaded=42846965

Note: The server is not using chunked encoding and Content-Length is accurate.

Expected Behavior

httpie should:

  • Trust Content-Length as the size of the compressed body.
  • Simply save the file without decompressing or misinterpreting the length.

This is what browsers, curl, and wget all do correctly.

Actual Behavior

httpie seems to assume Content-Length is the uncompressed size, leading it to flag the download as incomplete when it receives more data than expected.

Environment

  • HTTPie version: httpie 3.2.4
  • OS: macOS / Linux (reproduced in both)
  • Server: Custom Go server (Fiber), but same issue with static gzipped files

References

  • [RFC 9110 Β§ 8.6](https://www.rfc-editor.org/rfc/rfc9110#section-8.6):

    β€œThe Content-Length field indicates the number of octets in the body of the message after any content codings have been applied.”

Debug Info

http get "localhost:3000/metrics/admin/00:11:00:00:22:00/20250101T000000Z/download?format=json" Authorization:"Bearer ey....." --download --debug
HTTPie 3.2.4
Requests 2.32.4
Pygments 2.19.1
Python 3.13.5 (main, Jun 11 2025, 15:36:57) [Clang 17.0.0 (clang-1700.0.13.3)]
/opt/homebrew/Cellar/httpie/3.2.4_3/libexec/bin/python
Darwin 24.5.0

<Environment {'apply_warnings_filter': <function Environment.apply_warnings_filter at 0x10665cd60>,
 'args': Namespace(),
 'as_silent': <function Environment.as_silent at 0x10665cc20>,
 'colors': 256,
 'config': {'default_options': []},
 'config_dir': PosixPath('/Users/alex/.config/httpie'),
 'devnull': <property object at 0x106642110>,
 'is_windows': False,
 'log_error': <function Environment.log_error at 0x10665ccc0>,
 'program_name': 'http',
 'quiet': 0,
 'rich_console': <functools.cached_property object at 0x105a0e690>,
 'rich_error_console': <functools.cached_property object at 0x106630c00>,
 'show_displays': True,
 'stderr': <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>,
 'stderr_isatty': True,
 'stdin': <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>,
 'stdin_encoding': 'utf-8',
 'stdin_isatty': True,
 'stdout': <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>,
 'stdout_encoding': 'utf-8',
 'stdout_isatty': True}>

<PluginManager {'adapters': [],
 'auth': [<class 'httpie.plugins.builtin.BasicAuthPlugin'>,
          <class 'httpie.plugins.builtin.DigestAuthPlugin'>,
          <class 'httpie.plugins.builtin.BearerAuthPlugin'>],
 'converters': [],
 'formatters': [<class 'httpie.output.formatters.headers.HeadersFormatter'>,
                <class 'httpie.output.formatters.json.JSONFormatter'>,
                <class 'httpie.output.formatters.xml.XMLFormatter'>,
                <class 'httpie.output.formatters.colors.ColorFormatter'>]}>

>>> requests.request(**{'auth': None,
 'data': RequestJSONDataDict(),
 'headers': <HTTPHeadersDict('User-Agent': b'HTTPie/3.2.4', 'Authorization': b'Bearer eyJh...', 'Accept-Encoding': b'identity')>,
 'method': 'get',
 'params': <generator object MultiValueOrderedDict.items at 0x106a7d030>,
 'url': 'http://localhost:3000/metrics/admin/00:11:00:00:22:00/20250101T000000Z/download?format=json'})

HTTP/1.1 200 OK
Content-Disposition: attachment; filename="001100002200-20250101T000000Z.alog.json.gzip"
Content-Encoding: gzip
Content-Length: 5084527
Content-Type: application/json
Date: Thu, 10 Jul 2025 16:22:21 GMT

Downloading to 001100002200-20250101T000000Z.alog.json.gzip
Done. 42.8 MB in 00:0.07174 (597.2 MB/s)

http: error: Incomplete download: size=5084527; downloaded=42846965

kohnalex avatar Jul 10 '25 16:07 kohnalex