tornadoweb
Support streaming API for AsyncHTTPClient
Mostly like this: https://aiohttp.readthedocs.io/en/stable/client_quickstart.html#streaming-response-content
This could probably solve some usage like #157.
It is hard to control the process in streaming_callback, and it does not support async function as well.
A seemingly simple fix is to wrap calls here with code like:
class _HTTPConnection(httputil.HTTPMessageDelegate):
# ...
def data_received(self, chunk: bytes) -> Optional[Awaitable[None]]:
if self._should_follow_redirect():
# We're going to follow a redirect so just discard the body.
return None
if self.request.streaming_callback is not None:
# Should return either 'None' or a Future[None] to throttle future reads.
return self.request.streaming_callback(chunk)
self.chunks.append(chunk)
return None
then to make sure that places that call: delegate.data_received(chunk) do the dance of:
res = delegate.data_received(chunk)
if res:
await res # or yield...
Wouldn't it be a bug if this behavior is not be supported, even though the parity of:
@web.stream_request_body
class Handler(tornado.web.RequestHandler):
# ...
async def data_received(self, chunk):
# ....
is?
Additional library support (i.e. using httpx or similar) would work in some cases, but seeing that tornado has a curl client as well as this simple client already, it'd be nice to avoid additional dependencies for simple use-cases.
If it is any help, I created a PR here: https://github.com/tornadoweb/tornado/pull/3471 It is currently in draft, but (after checking the linter state and so forth), I can mark it ready for review.
