C# Windows app - Only One ICE Candidate

[mpogra]

Hello, We ar working on creating a streaming app to stream the audio of the system microphone to web browser, for the same we have created C@ app that uses this library (a really great and extensive lib you have created).

This seems to be working on local systems but as soon as I open the browser on any other system (or production) streaming does not work and the connection fails, though the stream is received on the browser from the C# app but it's not playing and after few seconds connection state is changed to failed. One thing I noticed is that C# app is only generating one ICE candidate whereas the browser is generating 5-6. I am using TURN servers from metered.ca for testing. Below is my code -

public void AddRTCClient(RTCOfferVO remoteOffer) {
            log.Debug("Remote offer received..."+remoteOffer);
            ThisConnectionBrowserID = remoteOffer.BrowserID;
            pc = new RTCPeerConnection(config);
            pc.onconnectionstatechange += Pc_onconnectionstatechangeAsync;
            pc.onicecandidate += Pc_onicecandidate;
            pc.onnegotiationneeded += Pc_onnegotiationneeded;
            pc.onsignalingstatechange += Pc_onsignalingstatechange;
            pc.OnReceiveReport += (re, media, rr) => log.Debug($"RTCP Receive for {media} from {re}\n{rr.GetDebugSummary()}");
            pc.OnSendReport += (media, sr) => log.Debug($"RTCP Send for {media}\n{sr.GetDebugSummary()}");
            //pc.GetRtpChannel().OnStunMessageReceived += (msg, ep, isRelay) => log.Debug($"STUN {msg.Header.MessageType} received from {ep}.");

            //MusicFile=Path.Combine(CommonApplicationData.Instance.ApplicationFolderPath, "test"
            //  audioSource = new AudioExtrasSource(new AudioEncoder(), new AudioSourceOptions () );
            log.Debug("Done setting up peer connection...");
            audioSource = new WindowsAudioEndPoint(new AudioEncoder(), this.DeviceIndex, this.DeviceIndex);
            audioSource.OnAudioSourceEncodedSample += pc.SendAudio;

            audioTrack = new MediaStreamTrack(audioSource.GetAudioSourceFormats(), MediaStreamStatusEnum.SendOnly);

            log.Debug("MediaStream created and added...");

            pc.addTrack(audioTrack);
            audioSource.OnAudioSourceEncodedSample += pc.SendAudio;
            pc.OnAudioFormatsNegotiated += (formats) => audioSource.SetAudioSourceFormat(formats.First());

            var of = new RTCSessionDescriptionInit { sdp = remoteOffer.sdp, type = RTCSdpType.offer };
            pc.setRemoteDescription(of);

           // var oo = new RTCOfferOptions { X_ExcludeIceCandidates}
            var offer = pc.createOffer();
            pc.setLocalDescription(offer);
            log.Debug("Sending local offer - \n" + offer.toJSON());
            RTCOfferVO vo = new RTCOfferVO { sdp = offer.sdp, DeskID = remoteOffer.DeskID, BrowserID = remoteOffer.BrowserID };
            SIOClient.Instance.EmitEvent("streamer-offer", vo);

        }

The ICE candidate I am getting is in C# app is mentioned below -

2024-10-18 19:21:10,057 [14] DEBUG WebRTCClient - Remote offer received...desk_streamer.src.model.RTCOfferVO 2024-10-18 19:21:12,562 [14] DEBUG WebRTCClient - Done setting up peer connection... 2024-10-18 19:21:12,806 [14] DEBUG WebRTCClient - MediaStream created and added... 2024-10-18 19:21:12,915 [14] DEBUG WebRTCClient - Pc_onsignalingstatechange - SignalingState: have_remote_offer ConnectionState: new, canTrickleIceCandidates:True 2024-10-18 19:21:13,046 [8] DEBUG WebRTCClient - Pc_onicecandidate - 1511 1 udp 659136 192.168.0.38 49458 typ host generation 0 2024-10-18 19:21:13,076 [14] DEBUG WebRTCClient - Pc_onsignalingstatechange - SignalingState: stable ConnectionState: new, canTrickleIceCandidates:True 2024-10-18 19:21:13,082 [14] DEBUG WebRTCClient - Sending local offer - {"type":"offer","sdp":"v=0\r\no=- 35803 0 IN IP4 127.0.0.1\r\ns=sipsorcery\r\nt=0 0\r\na=group:BUNDLE 0\r\nm=audio 9 UDP/TLS/RTP/SAVP 9 0 8 126\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:YCYD\r\na=ice-pwd:WWZBXLYDGZMHOOJRMMTBMJRS\r\na=fingerprint:sha-256 1A:7D:42:4D:53:EA:87:F7:21:30:F0:CF:76:5E:B2:7F:E7:A0:5B:DC:B7:67:2F:F8:B4:9B:B3:EB:8C:0B:8C:A3\r\na=setup:active\r\na=candidate:1511 1 udp 659136 192.168.0.38 49458 typ host generation 0\r\na=ice-options:ice2,trickle\r\na=mid:0\r\na=rtpmap:9 G722/8000\r\na=rtpmap:0 PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:126 telephone-event/8000\r\na=rtcp-mux\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=sendonly\r\na=ssrc:338556634 cname:4a55be4c-d73a-4775-9c7a-81b4844cd6d3\r\n"}

I think this is the reason (only one ICE with local IP) is causing it to fail. Please advise.

Thanks MP

[ha-ves]

I'm commenting for preliminary measures. highly similar with #861

host ice candidates are created per "connected" network interface in your host machine.

The other ones are usually srflx and relay which are to be determined when using a STUN/TURN server. Since you are using pc = new RTCPeerConnection(config);, I assume you are configuring with a STUN/TURN server.

This is a race condition as SIPSorcery internally doesn't do check/wait for finished ICE candidates gathering.

You can try waiting with while (pc.iceGatheringState != RTCIceGatheringState.complete); or any other mechanism on it that suits your needs (async usually complains on this).

[ha-ves]

Another way is to also signal pc.onicecandidate += Pc_onicecandidate; to the other peer.

[mpogra]

@ha-ves thanks for the quick reply, I am checking it.

One thing I noticed that no complete event is ever fired for Pc_iceGatheringState, it is always gathering. I am planning to send local offer to remote only when gathering is "complete" so that SDP will have all host in string.

Also i am using third party turn servers from (expressturn.com) for testing, strange thing is if both sender and receiver are on same wifi network it works all good but if i change network of any one of them, it just stuck. and eventually after few seconds browser PC state change to "failed". Any idea?

[ha-ves]

One thing I noticed that no complete event is ever fired for Pc_iceGatheringState, it is always gathering.

That's certainly weird, it will usually timeout after 10s or so (not sure the timing, haven't checked the code again).

Also i am using third party turn servers from (expressturn.com) for testing, strange thing is if both sender and receiver are on same wifi network it works all good but if i change network of any one of them, it just stuck. and eventually after few seconds browser PC state change to "failed". Any idea?

Can you check that TURN server works with the trickle ice from webrtc sample and gets both srflx & relay?

[mpogra]

@ha-ves I think in my case ICE servers were the issue, after changing to expressturn.com ICE servers, it's working fine across the networks and also getting Pc_iceGatheringState complete event as well, really that took time.

Thank you, this is resolved now but one problem ended and another started, now audio streaming sound (from windows microphone) is too sluggish, it is like someone is speaking in a fainted state of mind. I am using SIPSorceryMedia.Windows.WindowsAudioEndPoint that eventually uses Naudio . The AudioSamplingRates used by AudioEndPoint is 16K from the logs WindowsAudioEndPoint - Windows audio end point adjusting capture rate from 8000 to 16000. Below is my SDP -

v=0 o=- 22137 0 IN IP4 127.0.0.1 s=sipsorcery t=0 0 a=group:BUNDLE 0 m=audio 9 UDP/TLS/RTP/SAVP 9 0 8 126 c=IN IP4 0.0.0.0 a=ice-ufrag:EJLC a=ice-pwd:XJARIJJZBFVKXHUSKVJNWOGG a=fingerprint:sha-256 57:84:48:81:59:74:D1:C6:66:42:26:DF:48:C6:B2:4D:B9:E2:E0:8C:A7:3E:2E:3B:20:44:DA:CC:CC:52:12:FD a=setup:active a=candidate:1511 1 udp 659136 192.168.0.38 58286 typ host generation 0 a=candidate:5289 1 udp 659058 X4.1XX.1XX.X51 16524 typ relay raddr 0.0.0.0 rport 0 generation 0 a=candidate:4429 1 udp 659084 1XX.X1.XX.178 58286 typ srflx raddr 0.0.0.0 rport 0 generation 0 a=mid:0 a=rtpmap:9 G722/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:126 telephone-event/8000 a=ice-options:ice2,trickle a=rtcp-mux a=rtcp:9 IN IP4 0.0.0.0 a=sendonly a=ssrc:1029558980 cname:9ca65b8c-78ea-4071-8b27-b159098c68f8

Streaming is now all working but voice is very sluggish, could it be bit rate mentioned in SDP as 8000? Below are more details of device and format, WindowsEndPoint is using G722, first format -

 device is: Microphone (TONOR TM20 Audio Device)
 Format: G722, Codec: G722, ClockRate: 16000, RtpClockRate8000, FormatID:9, ChannelCount:1
 Format: PCMU, Codec: PCMU, ClockRate: 8000, RtpClockRate8000, FormatID:0, ChannelCount:1
 Format: PCMA, Codec: PCMA, ClockRate: 8000, RtpClockRate8000, FormatID:8, ChannelCount:1

 WindowsAudioEndPoint - Windows audio end point adjusting capture rate from 8000 to 16000.

My whole setup is now working fine now (that's to this wonderful library and help here) but now this small issue is troubling me. Please help.

Thanks.

[ha-ves]

Glad your ICE problem is fixed.

For the voice problem I'm not sure about it, I'm not familiar with the SIP part of this library.

But maybe some similar issues you can check with #707 and #1122

[sipsorcery]

Streaming is now all working but voice is very sluggish, could it be bit rate mentioned in SDP as 8000? Below are more details of device and format, WindowsEndPoint is using G722, first format -

See if it works if you restrict the format to PCM.

var windowsAudio = new WindowsAudioEndPoint(new AudioEncoder()); windowsAudio.RestrictFormats(format => format.Codec == AudioCodecsEnum.PCMU || format.Codec == AudioCodecsEnum.PCMA);

[sipsorcery]

Please feel free to re-open, or even better open a separate issue for the audio problem, if there is any more info.