openai
openai.Completion.acreate(stream=True) raises an Exception
See stack trace below
Traceback (most recent call last):
File "/Users/nuno/dev/concat/server-py/concat/conversation.py", line 75, in handle_conversation
await run
File "/Users/nuno/dev/concat/server-py/concat/agent/openai/openai.py", line 122, in __call__
msg = await ctx.stream_message_async(
File "/Users/nuno/dev/concat/server-py/concat/context.py", line 287, in stream_message_async
async for text in iterator:
File "/Users/nuno/dev/concat/server-py/concat/agent/openai/openai.py", line 60, in openai_completion_stream
iterator = await openai.Completion.acreate(*args, **kwargs, stream=True)
File "/Users/nuno/Library/Caches/pypoetry/virtualenvs/platform-api-30l3Kv3P-py3.10/lib/python3.10/site-packages/openai/api_resources/completion.py", line 45, in acreate
return await super().acreate(*args, **kwargs)
File "/Users/nuno/Library/Caches/pypoetry/virtualenvs/platform-api-30l3Kv3P-py3.10/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 230, in acreate
return (
TypeError: 'async_generator' object is not iterable
I believe this should be an async for in line 239 of engine_api_resource.py. Once I make that change, then I get this error
Traceback (most recent call last):
File "/Users/nuno/Library/Caches/pypoetry/virtualenvs/platform-api-30l3Kv3P-py3.10/lib/python3.10/site-packages/openai/api_requestor.py", line 645, in _interpret_response_line
data = json.loads(rbody)
File "/opt/homebrew/Cellar/[email protected]/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/opt/homebrew/Cellar/[email protected]/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/opt/homebrew/Cellar/[email protected]/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/nuno/dev/concat/server-py/concat/conversation.py", line 75, in handle_conversation
await run
File "/Users/nuno/dev/concat/server-py/concat/agent/openai/openai.py", line 122, in __call__
msg = await ctx.stream_message_async(
File "/Users/nuno/dev/concat/server-py/concat/context.py", line 287, in stream_message_async
async for text in iterator:
File "/Users/nuno/dev/concat/server-py/concat/agent/openai/openai.py", line 61, in openai_completion_stream
async for chunk in iterator:
File "/Users/nuno/Library/Caches/pypoetry/virtualenvs/platform-api-30l3Kv3P-py3.10/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 230, in <genexpr>
return (
File "/Users/nuno/Library/Caches/pypoetry/virtualenvs/platform-api-30l3Kv3P-py3.10/lib/python3.10/site-packages/openai/api_requestor.py", line 611, in <genexpr>
self._interpret_response_line(
File "/Users/nuno/Library/Caches/pypoetry/virtualenvs/platform-api-30l3Kv3P-py3.10/lib/python3.10/site-packages/openai/api_requestor.py", line 647, in _interpret_response_line
raise error.APIError(
openai.error.APIError: HTTP code 200 from API (
)
which appears to be caused by trying to json-decode the following string \n. Once I fix that (by doing eg. data = json.loads(rbody) if rbody.strip() else None, it appears the stream never finishes, ie. the next part never arrives
The final issue is in this line https://github.com/openai/openai-python/blob/4fee0da142f58307edb30111ed484d2d41e4811d/openai/api_requestor.py#L297
We exit the with statement before the response stream is consumed by the caller, therefore, unless we're using a global ClientSession, the session is closed (and thus the request) before it should be.
Thanks for reporting and debugging this @nfcampos, we'll take a look and hopefully we'll get a fix out shortly.
