Incompatible http streaming protocol with python requests

[Maxersh]

How to reproduce the issue:

import requests

token = "token"
cluster_id = "cluster_id"
url = f"https://mdb.api.cloud.yandex.net/managed-postgresql/v1/clusters/{cluster_id}:stream_logs"

with requests.get(url,
                  params={"serviceType":"POSTGRESQL"},
                  headers={"Authorization":f"Bearer {token}"},
                  stream=True
                  ) as response:
    for chunk in response.iter_content(chunk_size=None, decode_unicode=True):
        print(chunk)

The above code outputs the logs as they come in. However, an exception like the one below occurs when a timeout occurs.

Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 748, in _update_chunk_length
self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 443, in _error_catcher
yield
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 815, in read_chunked
self._update_chunk_length()
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 752, in _update_chunk_length
raise InvalidChunkLength(self, line)
urllib3.exceptions.InvalidChunkLength: InvalidChunkLength(got length b'', 0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/requests/models.py", line 760, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 623, in stream
for line in self.read_chunked(amt, decode_content=decode_content):
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 803, in read_chunked
with self._error_catcher():
File "/usr/lib64/python3.10/contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "/usr/lib/python3.10/site-packages/urllib3/response.py", line 460, in _error_catcher
raise ProtocolError("Connection broken: %r" % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/user/scripts/test.py", line 12, in <module>
for ind, chunk in enumerate(response.iter_content(chunk_size=None, decode_unicode=True)):
File "/usr/lib/python3.10/site-packages/requests/utils.py", line 549, in stream_decode_response_unicode
for chunk in iterator:
File "/usr/lib/python3.10/site-packages/requests/models.py", line 763, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

Apparently this only happens if any logs were output. If there are no logs, one message about stream timeout will be output and the program will end properly. I found a discussion of a similar error. Maintainer sad it's not a bug. This is a faulty implementation of the protocol on the server side.

[nikolaymatrosov]

It would help if you tried to use the gRPC version of the API. As Cloud API is gRPC first and the REST version is just a gateway, the gRRC endpoint should work.