tornado icon indicating copy to clipboard operation
tornado copied to clipboard

simple_httpclient: Timeout after redirect not raised properly

Open wxy8wxy opened this issue 4 years ago • 6 comments

Hello, My test codes are below:

import traceback from tornado import httpclient

def test(): client = httpclient.HTTPClient() url = "http://www.baidu.com/link?url=7rwodBCaetm-ogDMm0fCdkj-Crje0M21LIWnXbz3XxToow8VmZwXOZaEz1nBMJBKofatc2X8SlFXyNvv5D7HhWgpIB8MDi5R85IYL30OjDmW4URWPW5y4AIwkVKioJSxPPBzWJmmaO11fEEO__BLxIiiT1FcKw8asqqF7h1QupS" try: response = client.fetch(url) print(response) except: traceback.print_exc()

if name == "main": test()

It is suspended after got tornado.simple_httpclient.HTTPTimeoutError. The stack trace is

ERROR:asyncio:Exception in callback _HTTPConnection.finish..(<Future finis...uring request>) at /home/python3/.miniconda3/lib/python3.8/site-packages/tornado/simple_httpclient.py:664 handle: <Handle _HTTPConnection.finish..(<Future finis...uring request>) at /home/python3/.miniconda3/lib/python3.8/site-packages/tornado/simple_httpclient.py:664> Traceback (most recent call last): File "/home/python3/.miniconda3/lib/python3.8/asyncio/events.py", line 81, in _run self._context.run(self._callback, *self._args) File "/home/python3/.miniconda3/lib/python3.8/site-packages/tornado/simple_httpclient.py", line 664, in fut.add_done_callback(lambda f: final_callback(f.result())) tornado.simple_httpclient.HTTPTimeoutError: Timeout during request

How to fix it? Thanks!

Python 3.8.5 Tornado 6.1

Try to run it several times to get above error

wxy8wxy avatar Sep 18 '21 06:09 wxy8wxy

Did you try to open the URL in your web browser?

Python-37 avatar Sep 18 '21 08:09 Python-37

This looks like a bug when following redirects: some errors from the second request aren't getting raised correctly. This probably affects asynchronous usage as well as the synchronous HTTPClient, although I'm not certain. There are a couple of workarounds:

  • Install pycurl and run AsyncHTTPClient.configure("tornado.curl_httpclient.CurlAsyncHTTPClient") at the start of your program
  • Pass follow_redirects=False to fetch and handle the redirect yourself.

bdarnell avatar Sep 18 '21 13:09 bdarnell

AsyncHTTPClient also has the same error. The CurlAsyncHTTPClient works Thanks!

wxy8wxy avatar Sep 18 '21 17:09 wxy8wxy