`AzureOpenAI` does not take `model` parameter into account when creating a request

[AspadaX]

As I figured, the current approach is to use the deployment_id that the user sat when instantiating the AzureConfig, and it gets handled by the url() method provided by the Config trait.

impl Config for AzureConfig {
    fn headers(&self) -> HeaderMap {
        let mut headers = HeaderMap::new();

        headers.insert("api-key", self.api_key.expose_secret().parse().unwrap());

        headers
    }

    fn url(&self, path: &str) -> String {
        format!(
            "{}/openai/deployments/{}{}",
            self.api_base, self.deployment_id, path
        ) // It pieces things together here... 
    }

    fn api_base(&self) -> &str {
        &self.api_base
    }

    fn api_key(&self) -> &SecretString {
        &self.api_key
    }

    fn query(&self) -> Vec<(&str, &str)> {
        vec![("api-version", &self.api_version)]
    }
}

However, after I tried, the model parameter does not seem to work in the request creation part, nor the example had mentioned about deployment_id. This is handled in the Python library, of course.

The so-called deployment_id is confusing, as the official documents phrases it as deployment_name, as of this post.

Could you please make the model work just like the normal OpenAI approach?

[felix-schultz]

The model is always attached to a deployment_id / name. So the model parameter itself would never have an effect. Could you clarify what you mean by the python library handling it?

If I setup a endpoint with gpt-4o and I send the model gpt-3.5 I will still get gpt-4o per design.

[AspadaX]

The model is always attached to a deployment_id / name. So the model parameter itself would never have an effect. Could you clarify what you mean by the python library handling it?

If I setup a endpoint with gpt-4o and I send the model gpt-3.5 I will still get gpt-4o per design.

Apologies for the delayed response. Here's an example of how the Python SDK operates:

def main() -> None:
    # Optionally set the deployment_name/id here, but I usually leave it empty.
    client = AzureOpenAI(api_key="", api_version="", azure_endpoint="", azure_deployment="a_deployment_name")
    # This setup offers more flexibility, as a single client can be used for various deployments.
    result = client.chat.completion.create(messages=[], model="a_deployment_name")

Currently, async_openai doesn't support this functionality. To work with different deployments, I need to create multiple clients, which prevents sharing a single instance across a session. Creating many instances could potentially impact performance and not convenient.

[CloverOS]

I think what he meant is that when you want to change a model, deplopment_id(in url) will auto change by model in request params.I also do that (create multiple clients).

[Dragon-GCS]

In openai python sdk, they rewrite the base url with model name when build the request, see https://github.com/openai/openai-python/blob/main/src/openai/lib/azure.py#L65, and can ignore deployment when init the client, see https://github.com/openai/openai-python/blob/main/src/openai/lib/azure.py#L225. I've check the code of this crate, seems not easy to implement this with few changes.

[Dragon-GCS]

I think this issue can be closed now, azure support V1 API now, user can set base_url with https://YOUR-RESOURCE-NAME.openai.azure.com/openai/v1/ without deployment_id and api_version.