gateway Problems with fallback configuration

Problems with fallback configuration

Open rodrigoSantosZup opened this issue 10 months ago • 0 comments

What Happened?

When it's instantiated a client for portkey, and invoked the chat.completions.create function, we found some possible problems when we configure the fallback process.

No matter if i overload the custom_host field inside config payload. The custom host never changes when we trigger the endpoint to use other deployments.
i've already passed the azure api_version when the client was instantiated, but if i do not pass again on fallback config, the result will be a 404 not found because portkey tries to hit the model endpoint with undefined apiVersion.
With this configuration, i got the fallback, but it always happen, no matter the configuration to trigger only when receive a 429 status code.