TypeError: 'OpenAIResponse' object is not iterable with stream=True

[tianlichunhong]

Issue you'd like to raise.

For windows version with web server, my code of python is as below:

import openai

openai.api_base = "http://localhost:4891/v1"
#openai.api_base = "https://api.openai.com/v1"

openai.api_key = "not needed for a local LLM"

# Set up the prompt and other parameters for the API request
prompt = "Hello"

# model = "gpt-3.5-turbo"
#model = "mpt-7b-chat"
model = "stable-vicuna-13B.q4_2"

# Make the API request

response = openai.Completion.create(
    model=model,
    prompt=prompt,
    max_tokens=1024,
    temperature=0.28,
    top_p=0.95,
    n=1,
    echo=False,
    stream=True
)

I set stream=True, but it show the error:

Traceback (most recent call last):
  File "C:\Users\Administrator\test.py", line 18, in <module>
    response = openai.Completion.create(
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\openai\api_resources\completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\openai\api_resources\abstract\engine_api_resource.py", line 165, in create
    assert not isinstance(response, OpenAIResponse)
AssertionError

Suggestion:

Can someone tell me, how to use stream? Thank you!

[williamspindox]

same error for me, stream=True doesn't work, please enable this feature to get the response in chunks like in client.

[Pokebrouserkat]

I asked ChatGPT and it said it was to do with the api providing a full response instead of actually streaming it, so this is just streight-up a missing feature in the backend that causes the frontend to break. stream=False works, but it is slower so I would rather wait for streaming to work, or find a way to get it fixed.

I am on macOS version BTW

[Pokebrouserkat]

here is the error as it appears in Jupyter:

---

AssertionError Traceback (most recent call last) Cell In[78], line 17 14 openai.api_key = "" 16 # a ChatCompletion request ---> 17 response = openai.ChatCompletion.create( 18 model='open-something', # require using an open model by not passing a valid closed model in 19 messages=[ 20 {'role': 'system', 'content': "you are a test AI which should act like Mario from Nintendo"}, 21 {'role': 'user', 'content': "hello, what's up?"} 22 ], # so, it's messages instead of a prompt. I like that, I will try that nest time 23 temperature=0.8, # it was 0 by default but it responded with "@@@@@@@@@@@@@@@@" every time, so I picked 0.8 and it makes sense now 24 stream=True # finally, I get the messages early! 25 ) 27 for chunk in response: 28 print(chunk)

File ~/miniconda3/envs/torch/lib/python3.9/site-packages/openai/api_resources/chat_completion.py:25, in ChatCompletion.create(cls, *args, **kwargs) 23 while True: 24 try: ---> 25 return super().create(*args, **kwargs) 26 except TryAgain as e: 27 if timeout is not None and time.time() > start + timeout:

File ~/miniconda3/envs/torch/lib/python3.9/site-packages/openai/api_resources/abstract/engine_api_resource.py:165, in EngineAPIResource.create(cls, api_key, api_base, api_type, request_id, api_version, organization, **params) 153 response, _, api_key = requestor.request( 154 "post", 155 url, (...) 160 request_timeout=request_timeout, 161 ) 163 if stream: 164 # must be an iterator --> 165 assert not isinstance(response, OpenAIResponse) 166 return ( 167 util.convert_to_openai_object( 168 line, (...) 175 for line in response 176 ) 177 else:

AssertionError:

[boneanxs]

Same error when getting stream response

[bgcgarlipp]

Getting the same error, not sure how the test passes if the streaming test code provides the same error. Are you all using the Windows version?

model = "ggml-mpt-7b-chat.bin"
prompt = "Who is Michael Jordan?"
tokens = []
for resp in openai.Completion.create(
        model=model,
        prompt=prompt,
        max_tokens=50,
        temperature=0.28,
        top_p=0.95,
        n=1,
        echo=True,
        stream=True):
    tokens.append(resp.choices[0].text)

[niansa]

This should be solved by now! If it persists, please open a new, updated issue.

Please always feel free to open more issues as needed.

[JasonCoBA]

I am having the issue, with v2.5.4 TypeError: 'OpenAIResponse' object is not iterable

using this code:

import openai
openai.api_base = "http://localhost:4891/v1"
openai.api_key = "not needed for a local LLM"

model = "./mistral-7b-v0.1.Q2_K.gguf"
prompt = """
Good morning!
        """
tokens = []
for resp in openai.Completion.create(
        model=model,
        prompt=prompt,
        max_tokens=50,
        temperature=0.28,
        top_p=0.95,
        n=1,
        echo=True,
        stream=True):
    tokens.append(resp.choices[0].text)