requests icon indicating copy to clipboard operation
requests copied to clipboard

Bug/6294 zero bytes files are chunked

Open eivindt opened this issue 2 years ago • 4 comments

See bug #6294 for original bug report.

This PR provides a possible fix for this, reverting the change in #2896.

The change in #2896 definitely causes some bad side effects, since not all web servers handle "chunked" transfer encoding (i.e. some don't handle it, some don't handle it well).

Problem demonstration

Given a simple bottle application:

from bottle import run, put, request, default_app

app = default_app()

@app.put("/")
def receive_file():
    upload = request.body.read()
    for head, val in request.headers.items():
        print("%-30s:\t%s" % (head, val))
    clength = request.headers.get("Content-Length", "not-set")
    cencoding = request.headers.get("Transfer-Encoding", "not-set")
    return { 'size': len(upload), 'content-length': clength, 'transfer-encoding': cencoding }

if __name__ == '__main__':
    app.run(host='localhost', port=8880)

And a simple requests script:

import requests
import io
empty_obj = io.BytesIO(b'')
resp = requests.put('http://localhost:8880/', data=empty_obj)
print(resp.status_code)
if resp.status_code == 200:
    print(resp.json())

Working Scenario

$ python server.py
Bottle v0.12.25 server starting up (using WSGIRefServer())...
Listening on http://localhost:8880/
Hit Ctrl-C to quit.

Content-Length                :	
Content-Type                  :	text/plain
Host                          :	localhost:8880
User-Agent                    :	python-requests/2.31.0
Accept-Encoding               :	gzip, deflate, br
Accept                        :	*/*
Connection                    :	keep-alive
Transfer-Encoding             :	chunked
127.0.0.1 - - [03/Nov/2023 15:51:51] "PUT / HTTP/1.1" 200 65

Test script output:

200
{'size': 0, 'content-length': '', 'transfer-encoding': 'chunked'}

Failing Scenario

$ gunicorn -b localhost:8880 server:app
[2023-11-03 15:52:05 +0100] [1326049] [INFO] Starting gunicorn 21.2.0
[2023-11-03 15:52:05 +0100] [1326049] [INFO] Listening at: http://127.0.0.1:8880 (1326049)
[2023-11-03 15:52:05 +0100] [1326049] [INFO] Using worker: sync
[2023-11-03 15:52:05 +0100] [1326050] [INFO] Booting worker with pid: 1326050

Test script output:

400

The Original Issue

The change causing this problem in the first place was due to failing to upload data read from a subprocess pipe.

This patch adds a check to super_len that avoids returning length 0 for file handles that are not regular files.

eivindt avatar Nov 06 '23 08:11 eivindt

closing & reopening to trigger a new build

sigmavirus24 avatar Feb 23 '24 00:02 sigmavirus24

@eivindt would you be willing to fix the lint errors?

sigmavirus24 avatar Feb 24 '24 12:02 sigmavirus24

Give me a couple of days and I'll fix them.

eivindt avatar Feb 24 '24 13:02 eivindt

Sorry for the delay, there were some windows specific issues I couldn't easily fix without a windows pc.

eivindt avatar Mar 10 '24 15:03 eivindt