Michal Charemza comments

Results 69 comments of


                                            Michal Charemza

standalone dag processor gets stuck when over 1k dag files

Just in case anyone else stumbles on this - we hit a very similar issue, and worked around it by wrapping the dag processor command with `timeout`. So in our...

Slow request and not reading request body causes ConnectionResetError on large responses if keep-alive is disabled, even if worker timeout is disabled and in sync, async and threaded workers

Ah @pajod thanks for your reply. My suspicion is that something other than the default 30 second worker timeout is at play however. I can force it to hit the...

Slow request and not reading request body causes ConnectionResetError on large responses if keep-alive is disabled, even if worker timeout is disabled and in sync, async and threaded workers

(I've now amended the bug description to include having the worker timeout disabled)

Slow request and not reading request body causes ConnectionResetError on large responses if keep-alive is disabled, even if worker timeout is disabled and in sync, async and threaded workers

> Running Wireshark on the connection I do see an RST from server to client just before the error So this is rolling around my head. From some googling, it...

Slow request and not reading request body causes ConnectionResetError on large responses if keep-alive is disabled, even if worker timeout is disabled and in sync, async and threaded workers

Have now got a slightly more minimal example. A 3-line server (no Flask and no files) in test.py ```python def app(environ, start_response): start_response('200 OK', []) return [b'-' * 6553600] ```...

Slow request and not reading request body causes ConnectionResetError on large responses if keep-alive is disabled, even if worker timeout is disabled and in sync, async and threaded workers

Aha! From poking around around code, also found that the error occurs on gevent and gthread workers if keepalive is _disabled_, so ``` gunicorn --workers=1 --worker-class=gevent --keep-alive=0 test:app ``` or...

Slow request and not reading request body causes ConnectionResetError on large responses if keep-alive is disabled, even if worker timeout is disabled and in sync, async and threaded workers

I wonder if the fix to this would be to call ``` client.shutdown(socket.SHUT_RDWR) ``` Just before closing the socket, in both sync and async workers? From some googling, this might...

Slow request and not reading request body causes ConnectionResetError on large responses if keep-alive is disabled, even if worker timeout is disabled and in sync, async and threaded workers

Actually have found a variation of request code that shows the error as well. Don't need a sleep to delay sending the request body, as long as there is a...

Slow request and not reading request body causes ConnectionResetError on large responses if keep-alive is disabled, even if worker timeout is disabled and in sync, async and threaded workers

@michdan So it's been a while, but I suspect no, since: - As you say, that issue is only gthread and this one is wider - ... and also the...