sseclient icon indicating copy to clipboard operation
sseclient copied to clipboard

urllib3 response object doesn't expose an iterator

Open tiras-j opened this issue 7 years ago • 0 comments

So this is a curious one - the urllib3 response object inherits io.Base which does provide an iterator over itself, however this iterator doesn't invoke the response object stream() method which handles negotiating chunked responses.

This can be seen explicitly when strace'ing a process using the SSEClient, there will be tons of 1 byte recvfrom syscalls which certainly should not happen when parsing chunked encoding since the chunk-size should be the reads (until the next chunk etc).

The fix is pretty simple, change for chunk in self._event_source: to for chunk in self._event_source.stream(): Although I'm uncertain if this is portable to other response objects (although requests just uses urllib3 under the hood anyways).

If I get some time I present a more detailed example... speaking of which is there a good public event-stream API that would serve for examples? That would be convenient.

EDIT: A good quick example is to strace a process using SSEClient vs a direct curl against the same endpoint. The curl will use tremendously less CPU and the recvfrom's will be, well, chunky.

tiras-j avatar Aug 02 '17 19:08 tiras-j