Support for Azure OpenAI with custom domain (Azure API / corporate environment)

[sanzoghenzo]

Description

I'm not sure if you prefer to have a new issue or if I should add this to as a comment of a closed issue, so forgive me if I choose the wrong way 😅 ...

#3117 added Azure OpenAI support for AI Assistant based on the domain of the base_url option, which has to be .openai.azure.com.

My organization gave me (and my colleagues) access to an Azure OpenAI instance via an url with domain azure-api.net, so I cannot use it with marimo.

Suggested solution

reconsider the introduction of the api_type parameter to handle Azure OpenAI custom domains.

The rest of the url is the same, so the other parameters can still be parsed with the current logic

Alternative

• add a list of possible azure domains for the url parsing, but who knows which are they?
• Add an [ai.azure] section to handle the Azure OpenAI separately

Additional context

I'm also using Aider, so it would be great to be able to reuse the AZURE_API_BASE, AZURE_API_KEY and AZURE_API_VERSION envvars, but I understand this would be out of scope

[mscolnick]

hey @sanzoghenzo - that sounds like a reasonable change. would you be open to helping contribute this feature?

[sanzoghenzo]

I can try! do you prefer the api_type in the current [ai.openai] section or is it better to create a new [ai.azure] section?

[mscolnick]

I think it might be better to go with [ai.azure]. But we will still need to set which provider to use, so maybe we add [ai.provider_type] like here: https://github.com/marimo-team/marimo/issues/4318

[mscolnick]

Hey @sanzoghenzo, anything we can help with on this PR? If you want to put up a draft PR, we can take a look early or make contributions

[mscolnick]

This is now fixed by https://github.com/marimo-team/marimo/pull/6018 (which will be in the next release 0.15.0). Providers are encoded in the model ID: azure/gpt-4o, openai/gpt-4o

[sanzoghenzo]

Sorry for bringing this up so late, but I finally got time to test this and... it still doesn't work for us.

This is because our base URL as an extra path element

https://<subdomain>.azure-api.net/<group_name>/openai/deployments/gpt-4o/?api-version=2024-02-01

but the deployment name is taken from the fourth path item

https://github.com/marimo-team/marimo/blob/ccd8b379260feaadd594b4aa4b871e7f0f674209/marimo/_server/ai/providers.py#L606

This won't be a problem for us in a while (since we should obtain a new endpoint without the item one of these days), and maybe it was too much of an edge case, but I believe that the url parsing should be a bit more sound.

Something like this, maybe?

path_parts = parsed_url.path.split("/")
deployment_pos = next(i for i, itm in enumerate(path_parts) if itm == "deployments") + 1
deployment_name = path_parts[deployment_pos]