[BUG]: When running the video streaming track on Android, the video source cannot be transmitted

[wmRD01]

Package version

3.0.0-pre.7

Environment

* OS:Android
* Unity version:2022.3

Steps To Reproduce

1. import package WebRTC 3.0.0-pre.7
2. in unity Create Scene
3. Create video-related objects, such as Camera, RawImage...

Current Behavior

When the video track is running on Android, the video source cannot be sent I don't know what the problem is. It can run when editing, but after packaging into an App, the video cannot be sent to the service. The program code is as follows:

 peerConnection = new()
        {
            OnIceConnectionChange = state => { Debug.Log("IceConnectionState: " + state); },
            OnConnectionStateChange = state => { Debug.Log("ConnectionState: " + state); },
            OnIceGatheringStateChange = state => { Debug.Log("IceGatheringState: " + state); },
            // OnIceGatheringChange = state => { Debug.Log("IceGatheringState: " + state); },
            OnIceCandidate = state => { Debug.Log("OnIceCandidate: " + state); },
            OnDataChannel = state => { Debug.Log("OnDataChannel: " + state); },
            OnNegotiationNeeded = () => { Debug.Log("OnNegotiationNeeded: "); },
            OnTrack = state => { Debug.Log("OnTrack: " + state); },
            // OnRemoveTrack = state => { Debug.Log("OnRemoveTrack: " + state); },
        };
        videoTrack = _camera.CaptureStreamTrack(localView.texture.width, localView.texture.height);
        peerConnection.AddTrack(videoTrack);

        StartCoroutine(WebRTC.Update());

        var offer = peerConnection.CreateOffer();
        yield return offer;

        var offerDesc = offer.Desc;

        var opLocal = peerConnection.SetLocalDescription(ref offerDesc);
        yield return opLocal;

        var filteredSdp = "";

        foreach (string sdpLine in offer.Desc.sdp.Split("\r\n"))
        {
            if (!sdpLine.StartsWith("a=extmap"))
            {
                filteredSdp += sdpLine + "\r\n";
            }
        }

        Debug.LogWarning(filteredSdp);

        using (UnityWebRequest www = new(
                $"https://webrtcpush.tlivewebrtcpush.com/webrtc/v2/whip",
                UnityWebRequest.kHttpVerbPOST
            ))
        {
            www.uploadHandler = new UploadHandlerRaw(Encoding.ASCII.GetBytes(filteredSdp));
            www.downloadHandler = new DownloadHandlerBuffer();
            www.SetRequestHeader("Content-Type", "application/sdp");
            www.SetRequestHeader("Authorization", $"Bearer {webrtcUrl}");

            yield return www.SendWebRequest();

            if (www.result != UnityWebRequest.Result.Success)
            {
                Debug.LogWarning(www.error);
            }
            else
            {
                Debug.LogWarning(www.downloadHandler.text);

                var answer = new RTCSessionDescription { type = RTCSdpType.Answer, sdp = www.downloadHandler.text };
                var opRemote = peerConnection.SetRemoteDescription(ref answer);
                yield return opRemote;
                if (opRemote.IsError)
                {
                    Debug.LogWarning(opRemote.Error);
                }
                else
                {
                    // Debug.LogWarning(opRemote.Current);
                }
            }
        };

Expected Behavior

When starting stream video call,you can pull video

Anything else?

return version to package WebRTC 3.0.0-pre.6 can Publish Streaming

[gtk2k]

Since the WHIP signaling URL is https://webrtcpush.tlivewebrtcpush.com/webrtc/v2/whip it seems that you are using Tencent's service. https://www.tencentcloud.com/document/product/267/57042 According to this document, the supported codecs are H264, AV1, and HEVC. It is possible that the software codec VP8 has been selected, so why not try setting the codec to H264 only, connecting your PC (Unity Editor) and Android via P2P, and see if you can send video correctly from Android in H264? There is a sample of how to select a codec in this repository, so please refer to it. Also, please check if there is an error when you call SetRemoteDescription() on the Answer.

[wmRD01]

Sorry for seeing the message so late. Thank you for your reply. Our colleagues confirmed that when the "Sdp" parameter was printed out, it was H264. In addition, "There are examples of how to select codecs in this repository". Since the documentation is like a maze and it is difficult to find the target, can you guide me how to find it? Our use of this library actually did not follow the official documentation because they did not provide the SDK used by Unity (the streaming service is not other services), so we can only observe how the API is used from the Demo and implement it ourselves. Although in the end we chose to return to version 3.0.0-pre.6 so that it can be used normally and gave up tracking this issue, I am talking to you with a communication mentality.

[dntam00]

Hi @wmRD01 , did you find the solution? I'm experiencing the same phenomenon, the webrtc is connected but video is not transmitted from PC to mobile application.

[wmRD01]

@dangngoctam00 hi~you can use WebRTC 3.0.0-pre.6 Solving the problem

[AlexRadacki]

I can confirm that android builds running the Vulkan API won't transmit video data since WebRTC 3.0.0-pre.7. OpenGLES runs fine though.

[dntam00]

Hi @wmRD01 Thank your for your information. I haven't tried version WebRTC 3.0.0-pre.6 yet, but I'm aware of a potential issue of our code, the process of deleting webRTC is not correct, this could sometimes lead to internal coroutine crashes.

[wmRD01]

@dntam00 You're welcome, glad I could help you. If you don't have to use WebRTC 3.0.0-pre.7, you can choose to use 6

[wmRD01]

@AlexRadacki Thank you for the information you provided, but when using OpenGLES, Unity's RenderTextur cannot render the user's camera image. For example, when I use my phone to open the camera image, the image will be rendered to Unity's RenderTexture component, but it renders a transparent texture. . So I concluded from my testing that OpenGLES cannot render properly. Because this issue has troubled us for a very long time and consumed a lot of development funds, we finally decided to downgrade to pre.6 to solve the problem, because pre.6 can run Vulkan API normally. Although the performance is extremely poor, at least it works.

[AlexRadacki]

We've created an issue about this, and had it verified:

https://issuetracker.unity3d.com/issues/android-vulkan-the-video-is-not-streamed-when-it-is-added-with-rtcrtpsender-dot-addtrack-method

Please vote it up, to push the importance - thanks!