Spring AI Azure OpenAI DeploymentNotFound issue

[radhakrishna67]

Bug description spring-ai-azure-openai-spring-boot-starter reports issue about DeploymentNotFound ( it requires Azure model deployment name) with properties included spring.ai.azure.openai.api-key= spring.ai.azure.openai.endpoint=https://abc.openai.azure.com/ spring.ai.azure.openai.embedding-model=text-embedding-ada-002

com.azure.core.exception.ResourceNotFoundException: Status code 404, "{"error":{"code":"DeploymentNotFound", "message":"The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again."}}" at com.azure.core.implementation.http.rest.RestProxyBase.instantiateUnexpectedException(RestProxyBase.java:347) ~[azure-core-1.41.0.jar:1.41.0] at com.azure.core.implementation.http.rest.SyncRestProxy.ensureExpectedStatus(SyncRestProxy.java:130) ~[azure-core-1.41.0.jar:1.41.0] at com.azure.core.implementation.http.rest.SyncRestProxy.handleRestReturnType(SyncRestProxy.java:213) ~[azure-core-1.41.0.jar:1.41.0] at com.azure.core.implementation.http.rest.SyncRestProxy.invoke(SyncRestProxy.java:81) ~[azure-core-1.41.0.jar:1.41.0] at com.azure.core.implementation.http.rest.RestProxyBase.invoke(RestProxyBase.java:109) ~[azure-core-1.41.0.jar:1.41.0] at com.azure.core.http.rest.RestProxy.invoke(RestProxy.java:91) ~[azure-core-1.41.0.jar:1.41.0] at jdk.proxy2/jdk.proxy2.$Proxy75.getChatCompletionsSync(Unknown Source) ~[na:na] at com.azure.ai.openai.implementation.OpenAIClientImpl.getChatCompletionsWithResponse(OpenAIClientImpl.java:897) ~[azure-ai-openai-1.0.0-beta.3.jar:1.0.0-beta.3] at com.azure.ai.openai.OpenAIClient.getChatCompletionsWithResponse(OpenAIClient.java:294) ~[azure-ai-openai-1.0.0-beta.3.jar:1.0.0-beta.3] at com.azure.ai.openai.OpenAIClient.getChatCompletions(OpenAIClient.java:430) ~[azure-ai-openai-1.0.0-beta.3.jar:1.0.0-beta.3] at org.springframework.ai.azure.openai.client.AzureOpenAiClient.generate(AzureOpenAiClient.java:60) ~[spring-ai-azure-openai-0.2.0-20230908.193706-31.jar:0.2.0-SNAPSHOT] at org.springframework.ai.openai.samples.helloworld.simple.SimpleAiController.completion(SimpleAiController.java:21) ~[classes/:na] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na] at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-6.0.11.jar:6.0.11] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) ~[spring-web-6.0.11.jar:6.0.11] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.0.11.jar:6.0.11] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:884) ~[spring-webmvc-6.0.11.jar:6.0.11] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797) ~[spring-webmvc-6.0.11.jar:6.0.11] at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-6.0.11.jar:6.0.11] at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1081) ~[spring-webmvc-6.0.11.jar:6.0.11] at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:974) ~[spring-webmvc-6.0.11.jar:6.0.11] at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1011) ~[spring-webmvc-6.0.11.jar:6.0.11] at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903) ~[spring-webmvc-6.0.11.jar:6.0.11] at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564) ~[tomcat-embed-core-10.1.11.jar:6.0] at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885) ~[spring-webmvc-6.0.11.jar:6.0.11] at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) ~[tomcat-embed-core-10.1.11.jar:6.0] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:205) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-embed-websocket-10.1.11.jar:10.1.11] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-6.0.11.jar:6.0.11] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.0.11.jar:6.0.11] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-6.0.11.jar:6.0.11] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.0.11.jar:6.0.11] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109) ~[spring-web-6.0.11.jar:6.0.11] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.0.11.jar:6.0.11] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-6.0.11.jar:6.0.11] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.0.11.jar:6.0.11] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:166) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:341) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:894) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1740) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-embed-core-10.1.11.jar:10.1.11] at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na]

Environment spring-ai-azure-openai-spring-boot-starter : 0.2.0-SNAPSHOT spring boot: 3.1.2

Steps to reproduce Just run the application and access the endpoint /ai/sample

Expected behavior It has to connect to Azure OpenAI and get the response.

Minimal Complete Reproducible example https://github.com/rd-1-2022/ai-azure-openai-helloworld.git

[markpollack]

Thanks for such a great issue, with a repo example

[berjanjonker]

For Azure the 'Deployment Name' should be used and not the 'Model Name'. As a workaround you can use: ((AzureOpenAiClient)aiClient).setModel("deployment-name");

[markpollack]

We will look into it. We want to be portable and I don't know yet what google/amazon are doing wrt to their client interfaces in terms of introducing a new concept of 'deployment name'.

[markpollack]

We are getting back to this now. Thanks for your patience! What is planned to do to address this is to

1. Change the @ConfigurationProperties for Azure OpenAI to include the property called 'deployment'. This will set the Azure AI DeploymentName at application startup time for use with ChatClient to agree with how Azure AI exposes AI services in Azure AI Studio web site.
2. To change the deployment name, there will be the ability to pass options at runtime via a class named AzureOpenAIOptions. This will contain the property DeploymentName and will override the value set at application start time via @ConfigurationProperties. This is part of a larger 'support runtime options' effort that will be done across the code base for all model providers.

The Azure Open AI API has a difficult tension in trying to support both OpenAI and Azure OpenAI, which makes for awkward usage in the OpenAI case. For example, this javadoc comment shows the tension.

    public ChatCompletions getChatCompletions(String deploymentOrModelName,
        ChatCompletionsOptions chatCompletionsOptions) {

where deploymentOrModelNameis described as

deploymentOrModelName – Specifies either the model deployment name (when using Azure OpenAI) or model name (when using non-Azure OpenAI) to use for this request. chatCompletionsOptions – The configuration information for a chat completions request. Completions support a wide variety of tasks and generate text that continues from or "completes" provided prompt data.

Similar, in ChatCompletionOptions

    /**
     * Set the model property: The model name to provide as part of this completions request.
     * Not applicable to Azure OpenAI, where deployment information should be included in the Azure
     * resource URI that's connected to.
     *
     * @param model the model value to set.
     * @return the ChatCompletionsOptions object itself.
     */
    @Generated
    public ChatCompletionsOptions setModel(String model) {

For Spring AI, a user who wants to switch from Azure OpenAI to OpenAI we would suggest to change the dependency of the ChatClient to use and not to use the Azure OpenAI client so there isn't any mismatch in expectation or unclear API usage.

[markpollack]

If this change is made, it also needs to be done in the AutoConfiguration Properties

[markpollack]

Change AzureOpenAiEmbeddingOptions and AzureOpenAiChatOptions use of setModel to setDeploymentName as that matches Azure OpenAI terminology.

[markpollack]

Moving this to 0.8.1 for now as I feel there may unintended side effects of this change and would like it not to hold up the 0.8.0 release.