openlimit icon indicating copy to clipboard operation
openlimit copied to clipboard

ChatRateLimiterWithRedis hangs with request_limit=1

Open Youssefbenhammouda opened this issue 1 year ago • 1 comments

rate_limiter = ChatRateLimiterWithRedis(
    request_limit=1, token_limit=40000, redis_url="redis://localhost:6379"
)
chat_params = {
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello!"}],
}
with rate_limiter.limit(**chat_params):
    response = openai.ChatCompletion.create(**chat_params)

The code above is hanging at the context manager. The Redis server is running and everything is okay, I am setting the number of requests to 1, this has to allow to me execute the openai.ChatCompletion.create because the code is synchronous so it means I am only doing one request, but it hangs waiting indefinitely.

Youssefbenhammouda avatar Jun 13 '23 16:06 Youssefbenhammouda