openlimit
openlimit copied to clipboard
ChatRateLimiterWithRedis hangs with request_limit=1
rate_limiter = ChatRateLimiterWithRedis(
request_limit=1, token_limit=40000, redis_url="redis://localhost:6379"
)
chat_params = {
"model": "gpt-4",
"messages": [{"role": "user", "content": "Hello!"}],
}
with rate_limiter.limit(**chat_params):
response = openai.ChatCompletion.create(**chat_params)
The code above is hanging at the context manager.
The Redis server is running and everything is okay, I am setting the number of requests to 1, this has to allow to me execute the openai.ChatCompletion.create
because the code is synchronous so it means I am only doing one request, but it hangs waiting indefinitely.