How it handle errors

[redigits-team]

We are developing some projects that occasionally trigger a 429 error due to exceeding our account’s usage limits, either because the applications make requests too quickly or because many users are using the applications simultaneously. Does BricksLLM handle this error by slowing down the requests to the OpenAI API, perhaps using retries or caching mechanisms?

[spikelu2016]

this is not handled by bricks as for now

[redigits-team]

this is not handled by bricks as for now

And could you suggest me How to handle this?

[beat]

With a failover to another account ? https://github.com/bricks-cloud/BricksLLM/blob/main/cookbook/openai_with_azure_openai_failover.md