[BUG] Ice candidates sent before Answer/Offer

[Kawottex]

Hello, I am trying to implement WebRTC communication between my web implementation (JavaScript) and Unity, but I encounter a problem. Sometimes, when I try to send an offer or an answer on Unity side, it sends Ice candidate before sending the offer or the answer. I'll take the answer as an example:

• I do SetRemoteDescription
• I do CreateAnswer
• I do SetLocalDescription
• Some Ice candidates are sent
• I send the answer through signaling server

I don't know why, but before the end of SetLocalDescription coroutine, Ice candidates are send through OnIceCandidate callback of PeerConnection, creating an error on my web side (No remote description error, because the answer hasn't been received).

For information, I've got no problem on a web <-> web communication, neither on Unity <-> Unity communication, only web <-> Unity Here is a snippet of my Unity code when it receives an offer and it needs to create an answer, maybe there is something I do wrong (I've removed all the error verifications to simplify, but I've verified and there is no error that appear):

        private IEnumerator CreateAnswer(string _sdp)
        {
            RTCSessionDescription sessionDesc = new RTCSessionDescription
            {
                type = RTCSdpType.Offer,
                sdp = _sdp
            };
            RTCSetSessionDescriptionAsyncOperation remoteDescOp = peerConnection.SetRemoteDescription(ref sessionDesc);
            yield return remoteDescOp;

            RTCSessionDescriptionAsyncOperation answerOp = peerConnection.CreateAnswer(ref answerOptions);
            yield return answerOp;

            RTCSessionDescription desc = answerOp.Desc;
            RTCSetSessionDescriptionAsyncOperation localDescOp = peerConnection.SetLocalDescription(ref desc);
            yield return localDescOp; // Ice candidates have been sent before the end of localDescOp
           
            SignalingServer.Send(desc.sdp);
        }

Thank you, Kevin

[Kawottex]

Edit: I've made a temporary fix in my OnIceCandidate callback which wait that the offer/answer has been sent before sending ice candidates. It works well but I know that this isn't a correct way to setup a webrtc connection. I still don't know what could cause this issue. Moreover, the way I create answer and offer on Unity and on Javascript/web are pretty similar and worked well until I tested them together, cross-platform.

[karasusan]

@kawottex Mm, I know this behavior but I have never known it is different from Web API. We need to investigate this issue to fix it.

I would like to ask the question. Is the number of times sending candidates the same between the browser and Unity?

[Kawottex]

Hello, sorry for the delay. No, the number of ice candidates sent on both sides is different between browser and Unity, no matter which one send the offer or receive the answer.

[karasusan]

Thanks @Kawottex We are investigating this issue and it might be fixed for the next version.

[karasusan]

@Kawottex I reproduced the issue you reported in my environment. Thanks. I am checking how to fix this.

[karasusan]

The timing of firing RTCPeerConnection.OnIceCandidate callback is when calling RTCPeerConnection.SetLocalDescription() on Unity. This behaviour is same as Google Chrome.

I think there is a problem with the contents of candidates.

[azixMcAze]

Hi, I am having the same problem. In my case, it seems that RTCPeerConnection.OnIceCandidate is fired before the AsyncOp returned by SetRemoteDescription() is done.

var op = m_peerConnection.SetLocalDescription(ref remoteDescription);
yield return op; // OnIceCandidate will fire while waiting

[kannan-xiao4]

@azixMcAze The behavior is correct. Calling OnIceCandidate has nothing to do with the processing of SetLocalDescription.

[kannan-xiao4]

@Kawottex This issue report is by design, can I close this?

[giovanni-bertoncelli]

@kannan-xiao4 I don't think this behaviour is by design: by MDN specification adding an ice candidate before an offer has been received throws an OperationError, particularly on a offer with a new track. Reference: https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/addIceCandidate#exceptions.

[kannan-xiao4]

@giovanni-bertoncelli I referred those pages https://www.w3.org/TR/webrtc/#dom-peerconnection-setlocaldescription https://datatracker.ietf.org/doc/html/rfc8829#section-3.5.1

Note：As noted in [RFC8829] (section 5.9.), calling this method may trigger the ICE candidate gathering process by the ICE Agent.

Invoke gathering process means that the OnIceCandidate may also be fired.

That's right, an Exception will occur if adding an ice candidate before an offer has been received. So I think that handling is required on the user side.

[azixMcAze]

@kannan-xiao4 https://www.w3.org/TR/webrtc/#dom-peerconnection-addicecandidate specifies that addIceCandidate will return an InvalidStateError error if remoteDescription is null.

If remoteDescription is null return a promise rejected with a newly created InvalidStateError.

This is always what is happening when OnIceCandidate is fired before SetLocalDescription has returned and the unity code has had the opportunity to send its offer to the other webRTC pair.

To me, this behaviour is not expected and the Unity WebRTC users should not have to buffer the ICE candidate in order to prevent the error. It is not the case with the JS webRTC API in Chrome and Firefox.

On a side note, it seems that yield return pc.SetLocalDescription() will always takes 2 frames to complete and OnIceCandidate will always be fired 1 frame after caling SetLocalDescription. All this is indenpendent of the framerate (i.e. event with Application.targetFrameRate = 1).

I think a part of the problem lies in the async operation code of this package.

[giovanni-bertoncelli]

@azixMcAze I noticed too that the problem does not appear on browser...

[kannan-xiao4]

@azixMcAze @giovanni-bertoncelli Thank you for the details. We will check Unity Plugin behavior again.

memo: WRS-185

[Deepslient]

I get the same question,please help me

[Deepslient]

when the network is poor ,the problem is worser

[kannan-xiao4]

@Deepslient Please provide more information about your problem.

[Deepslient]

https://stackoverflow.com/questions/38198751/domexception-error-processing-ice-candidate @karasusan the problem is the same with the web describe

[kannan-xiao4]

@Deepslient Thanks checking this problem.

The OnInceCandidate behavior is undefined in web standards. Therefore, it works with Chromium implementation for firing OnIceCandidate. We follow the Chromium implementation, so this issue will not change as long as the behavior of the Chromium implementation does not change.

To be on the safe side, we check if Chromium BugTracker has reported a similar issue and see how BugTracker handles this issue.