[BUG] Azure OpenAI SDK: cannot use HTTP calls when using local endpoint

[sinedied]

Describe the bug I want to use a local OpenAI/Azure OpenAI endpoint to test my app, like Ollama, LM Studio or ollamazure.

The issue is that when using an endpoint that start with http://, for example http://localhost:4041, I get this error:

Key credentials require HTTPS to prevent leaking the key

Even though I'm using a dummy key when running locally.

Exception or Stack Trace

java.lang.IllegalStateException: Key credentials require HTTPS to prevent leaking the key.
    at com.azure.core.http.policy.KeyCredentialPolicy.processSync (KeyCredentialPolicy.java:110)
    at com.azure.core.http.HttpPipelineNextSyncPolicy.processSync (HttpPipelineNextSyncPolicy.java:53)
    at com.azure.core.http.policy.CookiePolicy.processSync (CookiePolicy.java:73)
    at com.azure.core.http.HttpPipelineNextSyncPolicy.processSync (HttpPipelineNextSyncPolicy.java:53)
    at com.azure.core.http.policy.AddDatePolicy.processSync (AddDatePolicy.java:50)
    at com.azure.core.http.HttpPipelineNextSyncPolicy.processSync (HttpPipelineNextSyncPolicy.java:53)
    at com.azure.core.http.policy.RetryPolicy.attemptSync (RetryPolicy.java:211)
    at com.azure.core.http.policy.RetryPolicy.attemptSync (RetryPolicy.java:224)
    at com.azure.core.http.policy.RetryPolicy.attemptSync (RetryPolicy.java:224)
    at com.azure.core.http.policy.RetryPolicy.attemptSync (RetryPolicy.java:224)
    at com.azure.core.http.policy.RetryPolicy.processSync (RetryPolicy.java:161)
    at com.azure.core.http.HttpPipelineNextSyncPolicy.processSync (HttpPipelineNextSyncPolicy.java:53)
    at com.azure.core.http.policy.AddHeadersFromContextPolicy.processSync (AddHeadersFromContextPolicy.java:67)
    at com.azure.core.http.HttpPipelineNextSyncPolicy.processSync (HttpPipelineNextSyncPolicy.java:53)
    at com.azure.core.http.policy.RequestIdPolicy.processSync (RequestIdPolicy.java:77)
    at com.azure.core.http.HttpPipelineNextSyncPolicy.processSync (HttpPipelineNextSyncPolicy.java:53)
    at com.azure.core.http.policy.HttpPipelineSyncPolicy.processSync (HttpPipelineSyncPolicy.java:51)
    at com.azure.core.http.policy.UserAgentPolicy.processSync (UserAgentPolicy.java:174)
    at com.azure.core.http.HttpPipelineNextSyncPolicy.processSync (HttpPipelineNextSyncPolicy.java:53)
    at com.azure.core.http.HttpPipeline.sendSync (HttpPipeline.java:138)
    at com.azure.core.implementation.http.rest.SyncRestProxy.send (SyncRestProxy.java:62)
    at com.azure.core.implementation.http.rest.SyncRestProxy.invoke (SyncRestProxy.java:83)
    at com.azure.core.implementation.http.rest.RestProxyBase.invoke (RestProxyBase.java:124)
    at com.azure.core.http.rest.RestProxy.invoke (RestProxy.java:95)
    at jdk.proxy4.$Proxy30.getChatCompletionsSync (Unknown Source)
    at com.azure.ai.openai.implementation.OpenAIClientImpl.getChatCompletionsWithResponse (OpenAIClientImpl.java:1500)
    at com.azure.ai.openai.OpenAIClient.getChatCompletionsWithResponse (OpenAIClient.java:310)
    at com.azure.ai.openai.OpenAIClient.getChatCompletions (OpenAIClient.java:677)
    at com.sample.azure.App.main (App.java:43)
    at org.codehaus.mojo.exec.ExecJavaMojo.doMain (ExecJavaMojo.java:385)
    at org.codehaus.mojo.exec.ExecJavaMojo.doExec (ExecJavaMojo.java:374)
    at org.codehaus.mojo.exec.ExecJavaMojo.lambda$execute$0 (ExecJavaMojo.java:296)
    at java.lang.Thread.run (Thread.java:1583)

To Reproduce Steps to reproduce the behavior:

1. Open this repo in Codespaces: https://github.com/sinedied/ollamazure/tree/java-samples
2. Open a terminal and run ollamazure to start an Azure OpenAI proxy
3. Open another terminal and run:

cd samples/java/azure-sdk
mvn compile exec:java

Code Snippet

        OpenAIClient client = new OpenAIClientBuilder()
            .credential(new AzureKeyCredential("123456"))
            .endpoint("http://localhost:4041")
            .buildClient();

        List<ChatRequestMessage> chatMessages = new ArrayList<>();
        chatMessages.add(new ChatRequestUserMessage("Say hello!"));
        
        ChatCompletions chatCompletions = client.getChatCompletions("gpt-4",
            new ChatCompletionsOptions(chatMessages));
        
        System.out.println( chatCompletions.getChoices().get(0).getMessage().getContent());

Expected behavior No error, or an option like .allowUnsecureHttpRequests(true) to suppress the error.

Screenshots

Additional information: Note that the calls works when NOT calling .credential() on the builder, but this is not an option: we want to be running the exact same code whether running locally or using Azure, and all frameworks based on this SDK (LangChain4J for example) do not have the option to not use any credential (which makes sense).

Setup (please complete the following information):

• OS: Debian bullseye
• IDE: VS Code
• Library/Libraries: [azure-ai-openai:1.0.0-beta.10]
• Java version: [21]
• App Server/Environment: none
• Frameworks: none

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• [x] Bug Description Added
• [x] Repro Steps Added
• [x] Setup information Added

[github-actions[bot]]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[alzimmermsft]

Thanks for filing this issue @sinedied.

At this time there isn't a good way to disable the requirement for https when using credentials. I'll follow-up with the team on an enhancement for this as local testing is a valid case for this to be turned off.

[bruno-oliveira]

Any updates?

[antonio-campagnaro]

In some cases, specifying the key isn’t required. For example, when running the solution in a Docker container, the key can be managed directly within the container’s configuration settings. Therefore, it should be possible to call the API over HTTP without needing to explicitly specify the key.

[crondab]

I'd also like to be able to send to HTTP endpoints. I am intending to run LiteLLM Proxy in my Kubernetes cluster. Our cluster uses HTTP for all services with Cilium (which uses the WireGuard protocol) to secure all network traffic. This is a total blocker for us using this client at the moment 😢

[crondab]

13 minutes later...

We're realised we can get around this restriction by setting the Authorization header manually in the client builder:

        return new OpenAIClientBuilder()
//                .credential(new KeyCredential("sk-1234"))
                .clientOptions(new ClientOptions().setHeaders(List.of(new Header("Authorization", "Bearer sk-1234"))))
                .endpoint(url)
                .buildClient();

[sunnyverma88]

Any updates on this one

[sigurdfalk]

We are running an AI gateway in k8s and a facing the same issues as @crondab