[QUERY] OpenAI: Hang while getting chat completions

[mnns]

Library name and version

Azure.AI.OpenAI 1.0.0-beta.6

Query/Question

For some reason, GetChatCompletionsAsync() is stuck and doesn't return at all. GetCompletionsAsync - works perfectly. The issue is I don't know how to specify system/users roles in it. Is there a way to use system roles with GetCompletionsAsync?

Thanks

Environment

.NET SDK: Version: 8.0.100-preview.5.23303.2 Commit: 3fe444af72

Runtime Environment: OS Name: Windows OS Version: 10.0.22000 OS Platform: Windows RID: win10-x64 Base Path: C:\Program Files\dotnet\sdk\8.0.100-preview.5.23303.2\

.NET workloads installed: There are no installed workloads to display.

Host: Version: 8.0.0-preview.5.23280.8 Architecture: x64 Commit: bc78804f5d

VS 2022

[github-actions[bot]]

Thank you for your feedback. This has been routed to the support team for assistance.

[navba-MSFT]

Adding Service team to look into this.

[timaiv]

What worked for me was changing the region (from US East to UK South).

[trrwilson]

Thank you for contacting us! That's very strange (and concerning) that you're observing different behavior across regions like that. Does the hang happen frequently against EastUS? If it does, would it be possible to share the query, code, and/or any affected request IDs?

The client library shouldn't hang as described, but there's likely an underlying issue with the response itself that should be investigated, too -- this is particularly true if changing regions improves things.

[joewichowski]

I can verify this is true. East US, hangs. I made a new one in Canada, worked just fine (same code).

[mnns]

Btw for me it hangs on openai. Com, non Azure

[GonzaloLebron]

Same problem here, in one environment is working fine (app service in West Europe) and Azure OpenAI is in West Europe too Other environment is in North Europe, and I get no response from GetChatCompletionsAsync() I will ask for creating a new Azure OpenAI in North Europe and connect to it in the environment having the problem to see if this fixes the issue.

[rogeliofrancisco]

I faced the same problem, I even tried with East US and North Central US, in both cases it keeps hanging while getting chat completions. Tried using GetChatCompletionsAsync and GetCompletionsAsync, neither worked for me. Have you solved the problem yet?

[moshekrup]

I faced the same problem in the Canada region (the getChatCompletions requests are stuck and don't return at all). Any updates here?

[jcaldwell4A]

Is it completely stuck or is it just taking a really long time?

I'm getting something similar. Using East US and C# I get throttled to 6 calls per minute. The quota is 60.

The first call is fine. Second, third. The fourth call gets delayed to 15 seconds. The next call takes 60 seconds. Which would certainly feel like it's hanging if I weren't timing it and patient.

If I use python on the same exact endpoint, I get 60 calls a minute no problem.

This is clearly an internal issue of the sdk rather than the endpoints.

[chrisclaytonrazor]

Seeing this same problem using Semantic Kernel which uses this call for completions internally.

[smcmanis]

Related issue: https://github.com/Azure/azure-sdk-for-net/issues/40726

[rpcontinuousdelivery]

GetChatCompletionsAsync is intermittently hanging for me at the moment and so I tried the Azure OpenAI "Chat playground" and it also appears to suddenly freeze half way through its response message. I'm using UK South on GPT 4 1106-preview. I'm using a prompt that is about 100 lines, 1,700 tokens. I've got my timeout set to an hour. The server seems to terminate the connection before the timeout expires. I've tried shorter timeouts, they seem to just expire. The same prompt worked for me yesterday with about 10 minutes of processing time.