microsoft-graph-comms-samples icon indicating copy to clipboard operation
microsoft-graph-comms-samples copied to clipboard

Call Terminated with Internal Error (subcode 1203003) on Bot Join Attempt (AKS sample)

Open bijaykush opened this issue 8 months ago • 5 comments

When attempting to connect a Microsoft Graph Calling Bot to a Teams meeting, the call transitions from establishing to terminated almost immediately. The resultInfo contains code 500, subcode 1203003, and a "Server Internal Error" message

Reproduction Steps:

The bot is built using AksSamples(Deprecated)

Hosted locally behind ngrok for signaling and media connectivity.

HTTPS forwarding is configured as: https://contoso.ngrok.io → http://localhost:9442

Media TCP port is forwarded via ngrok: tcp://3.tcp.ngrok.io:29661 → localhost:8445

Verified that telnet 3.tcp.ngrok.io 29661 works from an external network ✅

The call is initiated correctly by Teams. The bot receives the incoming call notification. After calling AcceptAsync, the call enters establishing state and then immediately goes to terminated.

State: Establishing

{
    "@odata.type": "#microsoft.graph.commsNotifications",
    "value": [
        {
            "@odata.type": "#microsoft.graph.commsNotification",
            "changeType": "updated",
            "resource": "/app/calls/0b005c80-5781-4207-9b53-e88ce86d0549",
            "resourceUrl": "/communications/calls/0b005c80-5781-4207-9b53-e88ce86d0549",
            "resourceData": {
                "@odata.type": "#microsoft.graph.call",
                "state": "establishing",
                "chatInfo": {
                    "@odata.type": "#microsoft.graph.chatInfo",
                    "threadId": "19:meeting_OTlmM2I3NzYtMWRhMC00ZjE1LTlkMjgtN2FhOThkYjZjNGNk@thread.v2",
                    "messageId": "0"
                },
                "meetingInfo": {
                    "@odata.type": "#microsoft.graph.organizerMeetingInfo",
                    "organizer": {
                        "@odata.type": "#microsoft.graph.identitySet",
                        "user": {
                            "@odata.type": "#microsoft.graph.identity",
                            "id": "d33b8ed0-2014-4212-898d-6ee59a8052c2",
                            "tenantId": "XXXXXXc3ea3c4431dc"
                        }
                    }
                },
                "callChainId": "46cec8e7-d5e8-40b0-989f-b693cde5bf6e"
            }
        }
    ]
}

State: Terminated

{
    "@odata.type": "#microsoft.graph.commsNotifications",
    "value": [
        {
            "@odata.type": "#microsoft.graph.commsNotification",
            "changeType": "deleted",
            "resource": "/app/calls/0b005c80-5781-4207-9b53-e88ce86d0549",
            "resourceUrl": "/communications/calls/0b005c80-5781-4207-9b53-e88ce86d0549",
            "resourceData": {
                "@odata.type": "#microsoft.graph.call",
                "state": "terminated",
                "resultInfo": {
                    "@odata.type": "#microsoft.graph.resultInfo",
                    "code": 500,
                    "subcode": 1203003,
                    "message": "Server Internal Error. DiagCode: 500#1203003.@"
                },
                "chatInfo": {
                    "@odata.type": "#microsoft.graph.chatInfo",
                    "threadId": "19:meeting_OTlmM2I3NzYtMWRhMC00ZjE1LTlkMjgtN2FhOThkYjZjNGNk@thread.v2",
                    "messageId": "0"
                },
                "meetingInfo": {
                    "@odata.type": "#microsoft.graph.organizerMeetingInfo",
                    "organizer": {
                        "@odata.type": "#microsoft.graph.identitySet",
                        "user": {
                            "@odata.type": "#microsoft.graph.identity",
                            "id": "d33b8ed0-2014-4212-898d-6ee59a8052c2",
                            "tenantId": "XXXXXXc3ea3c4431dc"
                        }
                    }
                },
                "callChainId": "46cec8e7-d5e8-40b0-989f-b693cde5bf6e"
            }
        }
    ]
}

Media Platform initialization Code:

Image

Ngrok:

Image

.Env

Image

Note: Same code and setup was working fine few weeks ago. I am trying to run locally.

Tagging @ssulzer, @ksikorsk, and @InDieTasten as I’ve seen your helpful input on similar issues — would really appreciate your insights on this one as well. Of course, anyone else who can provide guidance is more than welcome to jump in too.

bijaykush avatar Apr 21 '25 01:04 bijaykush

What version of the Skype Media and Graph Calls SDK are you running? The "official" AKS sample is deprecated due to outdated dependencies and lack of maintenance.

InDieTasten avatar Apr 21 '25 13:04 InDieTasten

Your Problem is regarding the Certificate, it might be valid for your personal ngrok domain but not for 3.tcp.ngrok.io. So you need to setup a CName on a domain you own. After that you need a valid CA signed certifcate for your own domain that points to ngrok. The HTTP endpoint can be "ignored" as it will validate authority by ngrok on the proxy.

1fabi0 avatar Apr 22 '25 13:04 1fabi0

What version of the Skype Media and Graph Calls SDK are you running? The "official" AKS sample is deprecated due to outdated dependencies and lack of maintenance.

@InDieTasten I have been using this deprecated sample from last 2-3 years and already updated the code to meet my existing business logic.

    <PackageReference Include="Microsoft.Graph.Communications.Calls" Version="1.2.0.850" />
    <PackageReference Include="Microsoft.Graph.Communications.Calls.Media" Version="1.2.0.850" />
    <PackageReference Include="Microsoft.Skype.Bots.Media" Version="1.19.0.25-alpha" />

I also tried the sample being maintained by you and getting the same error. Please let me know if I am missing anything.

bijaykush avatar Apr 25 '25 02:04 bijaykush

Your Problem is regarding the Certificate, it might be valid for your personal ngrok domain but not for 3.tcp.ngrok.io. So you need to setup a CName on a domain you own. After that you need a valid CA signed certifcate for your own domain that points to ngrok. The HTTP endpoint can be "ignored" as it will validate authority by ngrok on the proxy.

Hey @1fabi0, I’m a bit confused. Why would I need a custom domain and a CNAME pointing to 3.tcp.ngrok.io, when I’m already using a custom ngrok domain — for example, contoso.ngrok.io? I’ve used Let’s Encrypt to generate the required certificate and have it installed on my machine.

This exact setup used to work fine before — even without a real domain. Back then, I wasn’t using a reserved TCP address. I just ran ngrok until it gave me a 0.tcp.ngrok.io, then updated the port in my environment config before starting the project. Everything worked without issues.

The only difference now is that I’m using a reserved TCP address instead of 0.tcp.ngrok.io. Has something changed recently on Microsoft’s side regarding how these calls are handled or validated?

bijaykush avatar Apr 25 '25 02:04 bijaykush

@bijaykush

AFAIK, nothing has changed on the Microsoft platform. However, it should not work because the certificate you're loading from the bot app is not intendet for HTTP traffic (this is already done by the ngrok reverse proxy) - it's meant for the TCP endpoint.

If you use a custom Ngrok domain and create a certificate for that domain, it still won’t work. This is because the domain you're trying to secure with the certificate is the TCP endpoint (3.tcp.ngrok.io), but the SAN of your certificate is your custom Ngrok domain (e.g., contoso.ngrok.io).

These domains don’t match, so the certificate is invalid for the TCP connection.

You have two options:

  • Configure Ngrok so that your custom domain can handle TCP connections (not sure if this is supported).

  • Set up a domain that CNAMES to the TCP Ngrok address.

Also to note previously there were problems using a different tcp endpoint of ngrok than 0.tcp.ngrok.io but that was because it seemed like some traffic was blocked by ngrok, I don't know if this is still the case and part of this problem.

1fabi0 avatar Apr 25 '25 04:04 1fabi0

I’m facing the same problem, and it appears that multiple users have reported similar issues with this repository. Could you please share if there have been any updates or progress regarding this?

emirhanozcn avatar Nov 06 '25 05:11 emirhanozcn

@InDieTasten @1fabi0 Image

If you see now my ngrok is running with 0.tcp.ngrok.io with port 15254. I have config this into env variables AzureSettings__InstancePublicPort=15254 AzureSettings__CallSignalingPort=9441 AzureSettings__InstanceInternalPort=8445

Now, I generated the certificate using certbot for ngrok domain (xyz.ngrok.io) where you can see that https endpoints are being reached out from MS, in the screenshot attached above.

Still my call is getting terminated from establishing state instead of going to established state.

@InDieTasten @1fabi0 @ssulzer @ksikorsk Please let me know if you any idea to resolve this issue.

For Microsoft Team: Please use the following callId for debugging if you can help with this.

"callId": "2b005880-9530-4656-94c8-114f129bf965", "scenarioId": "ed5fe7be-3bf3-4d68-9433-a50d33a1247c",

emirhanozcn Were you able to resolve this?

bijaykush avatar Nov 08 '25 05:11 bijaykush

hi @bijaykush A couple things to try:

  1. Please update to the newer media SDK, version 1.32: https://www.nuget.org/packages/Microsoft.Skype.Bots.Media/1.32.0.70-preview

  2. I see your bot is running on .NET Framework. There were important advisories in the media SDK's readme.txt for bots on .NET Framework which were noted for versions 1.28 and 1.31:

  • Please note the following important addendum to the Setup Instructions given in the release notes from SDK 1.28. The "netsh http add sslcert" command should also specify a "clientcertnegotiation=enable" parameter, otherwise a call setup failure may occur; e.g.,

    Bind ssl cert to the IP/Port and enable client certificate negotiation netsh http add sslcert ipport=%InstanceIpAddress%:%InstanceMediaControlPort% "appid={00000000-0000-0000-0000-000000000001}" cert=%ServerCertificate% clientcertnegotiation=enable

ssulzer avatar Nov 08 '25 07:11 ssulzer

@ssulzer Based on your suggestion, I did following changes

  1. Updated SDK to latest <PackageReference Include="Microsoft.Graph.Communications.Calls" Version="1.2.0.15690" /> <PackageReference Include="Microsoft.Graph.Communications.Calls.Media" Version="1.2.0.15690" /> <PackageReference Include="Microsoft.IdentityModel.Clients.ActiveDirectory" Version="5.2.9" /> <PackageReference Include="Microsoft.IdentityModel.Protocols.OpenIdConnect" Version="8.14.0" /> <PackageReference Include="Microsoft.NETCore.Platforms" Version="7.0.4" /> <PackageReference Include="Microsoft.Owin.Host.HttpListener" Version="4.2.2" /> <PackageReference Include="Microsoft.Owin.Hosting" Version="4.2.2" /> <PackageReference Include="Microsoft.Skype.Bots.Media" Version="1.32.0.70-preview" />

  2. Executed netsh http add sslcert with "clientcertnegotiation=enable" parameter

Image
  1. Ngrok running as HTTPS is working fine:
Image
  1. Telnet to 8445 is fine

Still I am getting this error with state directly moving to Terminated from Establishing

CallId: 30005b80-28f4-446a-af3f-7a1fd3c6e84b
Error: Server Internal Error. DiagCode: 500#1203002.

Anything else I need to check?

bijaykush avatar Nov 11 '25 00:11 bijaykush

Created a custom dns with GoDaddy and pointed CNAME to 0.tcp.ngrok.io and generated certificate for dns using certbot/letsencrypt and installed on localmachine. Bound the certificate to port 8445(in my case), and used dns for ServiceFqdn and another ngrok domain for signaling. And was able to run locally.

But previously, I did not use any custom DNS, all was working with ngrok domain only for both signaling and media endpoint.

bijaykush avatar Nov 17 '25 02:11 bijaykush