No rate limit error when using asynchronous calls for the chatgpt api

[userrand]

Describe the bug

When using the ChatGPT api with asynchronous calls, I do not receive rate error limits, instead the code seems stuck.

To Reproduce

Here are 2 example codes that use asynchronous calls.

Note that I learned about asynchronous calls for this purpose and so have little experience.

1 Without the Python wrapper

import asyncio
from aiohttp import ClientSession

openai_api_key = "key_here"

headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer " + openai_api_key,
}
url = "https://api.openai.com/v1/chat/completions"

async def ChatBatch(prompts,temperature=0,model= "gpt-3.5-turbo") :

    async def Chat_Result(session, prompt, url=url, headers=headers):
        data = {"model": model,
            "messages": [{"role": "user", "content": prompt}],
            "temperature": temperature
        }
        async with session.post(url, headers=headers, json=data) as result:
            return await result.json()

    async def get_response():
        results = []
        async with ClientSession() as session:
            for prompt in prompts:
                result = Chat_Result(session, prompt)
                results.append(result)
            return await asyncio.gather(*results)


    # return await get_response()
    return [response['choices'][0]['message']['content'] for response in await get_response()]

test :

await ChatBatch(["Hello how are you ?"]*30)

2 With the Python Wrapper

import openai
import asyncio
from aiohttp import ClientSession

openai.api_key = "key_here"


openai.aiosession.set(ClientSession())

async def create_completion(prompt="How are you ?"):
    return await openai.ChatCompletion.acreate(messages=[{"role": "user", "content": prompt}], model="gpt-3.5-turbo")

async def main():

    return await asyncio.gather(*[create_completion() for _ in range(14)])

answers=await main()

await openai.aiosession.get().close()

answers

Code snippets

No response

OS

Linux

Python version

Python 3.9.12

Library version

openai v0.27

[JakeWinter]

I also am experiencing this error. Sometimes works and doesn't stall, then sometimes will stall.

[userrand]

I also am experiencing this error. Sometimes works and doesn't stall, then sometimes will stall.

Yes yesterday was quite difficult. I realized that it stalls even without the asynchronous calls and I do not seem to receive rate limit errors. Today things seemed to move more smoothly.