feat (Azure): support Azure deployment selection

[nanaya-tachibana]

I'd like to add the support of Azure deployment selection to give the same model switching experience when using Azure openAI service.

This patch probably won't break anything when CUSTOM_MODELS is not set. User can use - to hide all internal models and use + to add azure deployments in CUSTOM_MODELS.

• When there is a request for an available model, replace the deployment in the AZURE_URL.
• When there is a request for an unavailable model, for example the summary scenario, use the default deployment in the AZURE_URL.

[H0llyW00dzZ]

Add both for openai aswell not only azure. it will better because in future, its not only text models based,(eg tts,dalle)

[Vigilans]

Also take for reference: https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3206#issuecomment-1807525917 and https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3206#issuecomment-1807529197.

[H0llyW00dzZ]

That's a reference summarizing how models should be selected depending on the models chosen by users for Azure.

here is the example

// fix known issue where summarize is not using the current model selected
function getSummarizeModel(currentModel: string, modelConfig: ModelConfig) {
  // should be depends of user selected
  return currentModel.startsWith("gpt") ? modelConfig.model : currentModel;
}

then simply how call getSummarizeModel

const sessionModelConfig = this.currentSession().mask.modelConfig;
const topicModel = getSummarizeModel(session.mask.modelConfig.model, sessionModelConfig);

[nanaya-tachibana]

Also take for reference: #3206 (comment) and #3206 (comment).

From my current understanding of the code, probably an advanced configuration is need to achieve that. Current CUSTOM_MODELS mechanism is not enough.

For example:

[
  {
    "api_provider": "openai",
    "base_url": "https://api.openai.com",
    "api_key": "XXXXXX",
    "models": [...]
  },
  {
    "api_provider": "openai",
    "base_url": "https://proxy.foo.bar",
    "api_key": "XXXXXX",
    "models": [...]
  },
  {
    "api_provider": "azure",
    "base_url": "https://{resource-name-1}.openai.azure.com/openai/deployments/",
    "api_key": "XXXXXX",
    "models": [...]
  },  
  {
    "api_provider": "azure",
    "base_url": "https://{resource-name-2}.openai.azure.com/openai/deployments/",
    "api_key": "XXXXXX",
    "models": [...]
  }
]

[nanaya-tachibana]

Add both for openai aswell not only azure. it will better because in future, its not only text models based,(eg tts,dalle)

I didn't quite get your point. It seems openAI APIs are quite consistent. All APIs share the same base url and model is passed as a request parameter. There is no need to generate url based on model.

I saw there is a PR working on TTS https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3258#issue-1991354310. let me check how the server side will be implemented and adapt some changes if possible.

[H0llyW00dzZ]

Add both for openai aswell not only azure. it will better because in future, its not only text models based,(eg tts,dalle)

I didn't quite get your point. It seems openAI APIs are quite consistent. All APIs share the same base url and model is passed as a request parameter. There is no need to generate url based on model.

I saw there is a PR working on TTS #3258 (comment). let me check how the server side will be implemented and adapt some changes if possible.

this one When there is a request for an unavailable model, for example the [summary scenario](https://github.com/Yidadaa/ChatGPT-Next-Web/blob/9da455a7eabcf22c3cbdb8bfe0ec3203a869e4e9/app/store/chat.ts#L85), use the default deployment in the AZURE_URL. actually this - > https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3260#issuecomment-1809728956

sorry I was replying on phone that first comments

[Vigilans]

Also take for reference: #3206 (comment) and #3206 (comment).

From my current understanding of the code, probably an advanced configuration is need to achieve that. Current CUSTOM_MODELS mechanism is not enough.

For example:

[
  {
    "api_provider": "openai",
    "base_url": "https://api.openai.com",
    "api_key": "XXXXXX",
    "models": [...]
  },
  {
    "api_provider": "openai",
    "base_url": "https://proxy.foo.bar",
    "api_key": "XXXXXX",
    "models": [...]
  },
  {
    "api_provider": "azure",
    "base_url": "https://{resource-name-1}.openai.azure.com/openai/deployments/",
    "api_key": "XXXXXX",
    "models": [...]
  },  
  {
    "api_provider": "azure",
    "base_url": "https://{resource-name-2}.openai.azure.com/openai/deployments/",
    "api_key": "XXXXXX",
    "models": [...]
  }
]

I think your proposed schema is quite good. May further add "name" field to it, to give each model group a semantic meaning, e.g. OpenAI, OpenAI Proxy, Azure JP, Azure US, etc.

Then in chat model selection window, we can group the models according to their providers:

• OpenAI
  • <Include all default OpenAI models (An option that could be disabled)>
  • <Some other custom models>
• OpenAI Proxy (e.g. api2d)
  • gpt-3.5-turbo
  • gpt-4
• Azure JP (no gpt-4 deployment currently available, but good network quality)
  • gpt-3.5-turbo
  • gpt-3.5-turbo-16k
• Azure US (providing more models, including gpt-4, whisper, etc.)
  • gpt-3.5-turbo
  • gpt-3.5-turbo-16k
  • gpt-4
  • gpt-4-32k
• Claude
  • <Include all Claude models>
• LLAMA
  • <Include all llama models>

Same model name could be repeatedly used in different groups, so users can decide which provider to use for gpt-3.5-turbo model at preference.

[nanaya-tachibana]

Add both for openai aswell not only azure. it will better because in future, its not only text models based,(eg tts,dalle)

I didn't quite get your point. It seems openAI APIs are quite consistent. All APIs share the same base url and model is passed as a request parameter. There is no need to generate url based on model. I saw there is a PR working on TTS #3258 (comment). let me check how the server side will be implemented and adapt some changes if possible.

this one When there is a request for an unavailable model, for example the [summary scenario](https://github.com/Yidadaa/ChatGPT-Next-Web/blob/9da455a7eabcf22c3cbdb8bfe0ec3203a869e4e9/app/store/chat.ts#L85), use the default deployment in the AZURE_URL. actually this - > #3260 (comment)

sorry I was replying on phone that first comments

Add both for openai aswell not only azure. it will better because in future, its not only text models based,(eg tts,dalle)

I didn't quite get your point. It seems openAI APIs are quite consistent. All APIs share the same base url and model is passed as a request parameter. There is no need to generate url based on model. I saw there is a PR working on TTS #3258 (comment). let me check how the server side will be implemented and adapt some changes if possible.

this one When there is a request for an unavailable model, for example the [summary scenario](https://github.com/Yidadaa/ChatGPT-Next-Web/blob/9da455a7eabcf22c3cbdb8bfe0ec3203a869e4e9/app/store/chat.ts#L85), use the default deployment in the AZURE_URL. actually this - > #3260 (comment)

sorry I was replying on phone that first comments

Oh, that makes sense to me. Thanks

[hansh0801]

i also has same issue on it.

[shendongming]

https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3344

I have already created a PR (Pull Request), which has resolved the issue.

Two changes:

1. Added a new configuration item: AZURE_OPENAI_MODEL_MAPPER
2. Extended the configuration item: AZURE_URL

The changes are as follows:

AZURE_URL (optional)

Example: https://{azure-resource-url}/openai/deployments/{deploy-name}

Example: https://xxx.openai.azure.com/openai/deployments/{deploy-name}

Azure deploy url.

If {deploy-name} is using the template mode, then it will automatically replace the path based on the model selected by the client. If your model name is different from the deployment name, then you need to set the AZURE_OPENAI_MODEL_MAPPER parameter.

AZURE_OPENAI_MODEL_MAPPER (optional)

Default: Empty Example: gpt-3.5-turbo=gpt-35-turbo means map gpt-3.5-turbo to gpt-35-turbo

If you are deploying ChatGPT using Azure OpenAI, it is recommended to set the AZURE_OPENAI_MODEL_MAPPER. The session summarization feature relies on the gpt-3.5-turbo model, unless the name of your Azure deployment is the same as it.