Token limit

[szym1998]

Issue Description

Problem: When using the new OpenAI library in my asynchronous application, I've encountered an issue related to rate limiting.

Description: It appears that when I run my asynchronous application, even just once, the rate limiter starts to restrict requests, preventing it from going through. This issue arises when I set the token limit to 90,000. However, when I increase the token limit to 900,000, the request go through without. It's important to note that my system message, user input, and response typically comprise only around 2,700 tokens in total.

Steps to Reproduce:

Install the OpenAI library 1.1 i think (the latest one) Set the token limit to 90,000. Run your asynchronous application. Observe the rate limiter restricting requests. Expected Behavior:

Requests should not be rate-limited when the token limit is set to 90,000, given that the total token count is well below this limit. Actual Behavior:

The rate limiter appears to limit requests, even when the token limit is set to 90,000.