[BUG] Render stream is blank for non "localhost" IP

[tkbala]

Describe the bug I am trying to run the web browser input scene and get it streamed to the video player sample web client. It is working with localhost, but not when the web browser client is opened on a different computer. The webpage loads up, but the stream is just black.

To Reproduce Steps to reproduce the behavior:

1. Start the webserver using the specified port

2. In the webbrowserInput scene in Unity, set the port in the render streaming game object to the desired port and start the program 1

3. On the remote computer open the video player sample and click play

Expected behavior A working render stream video .

Environment (please complete the following information):

• OS: Both Computers use Windows
• Browser : Both computers use chrome browser
• Unity Version 2019.4.26f1
• Package version 3.0.1

[cvachha]

I am having the same issue. Is there any fix?

[karasusan]

@cvachha @tkbala Is the signaling process completed? Can you disable the firewall to check ports opening or not?

[cvachha]

I have tried disabling the firewall (there does not seem to be a difference when I turn it on or off) and I have noticed that when I click play on the web client videoplayer on the remote machine, the Unity console prints 10 new debug.log statements and indicates that the signaling process is connected “Signaling: HTTP connected...”

However, on the remote machine, the render stream is still empty and the input buttons (“Light off”) do not work. If I run the web client on my local machine, the similar debug.log statements show up in the unity console but the render stream shows up and input works.

I am running the webserver on the same port number of the signaling URL and on the remote machine, The remote machine is on a different network (using a mobile hotspot) and port forwarding into the same port into the network I am running the render streaming server in.

[cvachha]

To follow up on my previous post, I recently created a TURN server (run on GCP and following the instructions provided in the Render Streaming documentation using coturn) and I am still getting the same issue (blank video player, and getting messages in the Unity console that there has been a connection). On my local computer the stream runs properly. I am using Unity 2019.4. I am running this in a new Unity project with the render streaming “WebBrowserInput” sample.

I am unsure if the TURN server is working as intended. On Trickle ICE, I am getting an error 701. In my signaling in Unity, I am putting localhost with a different port number from “3478”, and putting the “ice server” as "turn:[external IP of VM from GCP]:3478?transport=tcp"

In GCP Firewall, I am allowing the external IP address and tcp/udp ports (I am also allowing HTTP traffic in the VM instance details). My VM is a f1-micro running ubuntu-1604 and I have enabled the TURN server through coturn.

I am unsure how to proceed and am not sure if my TURN server is working correctly, or if the render streaming package is not working correctly.

Thank you.

[karasusan]

@cvachha Thanks for explaining details the issue. We need to know the TURN server works correctly without render streaming.

The simplest way is using chrome browser and WebRTC sample here.

the document for checking connection verification is here. https://docs.unity3d.com/Packages/[email protected]/manual/turnserver.html#connection-verification

[karasusan]

@tkbala Are there your PCs in the same network? Is there a same issue for all ports? How about other samples? Is this issue occurred for browsers only?

[karasusan]

@tkbala @cvachha Hi, could you answer questions above?

[Haohua-Lyu]

Hi, I encountered the same issue and possibly identified why this happens. So when using Http signaling, I found that the unity end sent 3-4 candidates, each with a different IP address (local, public, etc). In our case, the first candidate Unity sends out is always the local IP, and somehow the web browser is "hooked" to the first candidate, which includes a local IP address. Thus the streaming only works when the Unity app and the web client are in the same LAN, but not across WAN. I was able to circumvent this by switching to ws signaling.

[cvachha]

The PCs are not in the same network and the issues happen with different ports. This issue is present in the samples that include the videoplayer, where it is blank in the browsers and this seems to happen when using HTTP signaling.

[karasusan]

@Haohua-Lyu @cvachha Sure, we will check the signaling process across WAN to reproduce it.

[karasusan]

memo: URS-369

[scode-cmd]

This happens even with websocket signaling. Same issue, signaling and streaming works just fine when both client and server are connected to the same network, but render stream is blnk when they are on different network (but signaling works fine)

[karasusan]

@scode-cmd Sure. We will check them.

[karasusan]

@tkbala @scode-cmd @Haohua-Lyu @cvachha

I am testing signaling and streaming across WAN and I found the error when using websocket. It could be same this issue. Is there a console log on your browser?

There is the error log but streaming is working correctly. It's just a guess, but the cause of the issue might be NAT. In most cases for streaming on WAN, we need to use TURN/STUN server.

[karasusan]

@tkbala @scode-cmd @Haohua-Lyu @cvachha Hi, have you solved the issue using TURN/STUN server?

[Haohua-Lyu]

Hi, we didn't encounter this bug after shifting to ws signaling, but I do remember we had TURN/STUN servers in place when using HTTP signaling. Once we changed to ws signaling (with no changes on TURN/STUN), the streaming started to work. Hope this helps!

[eaberman]

I am having this issue as well, even using web sockets and a working TURN server.

I have deployed the Unity Render Streaming Web App on an EC2 Instance on AWS, as well as a TURN server on an EC2 Instance on AWS. I verified that the TURN server is working using Trickle ICE.

The web stream ALWAYS works when I am on the same network as the browser I have my web app on, but NEVER when I am not on the same network.

My Unity console and the requests I see on the web browser using developer tools show that SDP requests are exchanged, and ICE candidates are sent and received (including a relay candidate on TURN server), yet the stream remains blank on the Unity Video Receiver page in the browser.

I am using Unity Render Streaming 3.1.0-exp6, and Unity 2021.3.17f1.

Can you please advise me on any updates or ways to fix this issue?

[karasusan]

@eaberman You need to check whether the cause of issue is your network environment. First of all, you need to replicate the issue with browsers like google chrome, not using Unity. If you can replicate it without Unity, the cause of this issue should be your network environment.

[eaberman]

Hi @karasusan! Thank you for responding to me.

I believe I was able to replicate the problem without Unity. I access the Web App via Google Chrome, and have built the Unity app with the Render Streaming camera w/signaling manager and broadcast components to an iOS device, and tried to connect. I am able to replicate the issue.

Using Chrome developer tools I have gained more clarity on the problem:

• I can observe the Web App sending ICE Server candidates of type host, srflx, and relay to the Unity iOS app
• I can observe the Unity iOS app sending two candidates of type host, but never of any other type

I would expect the Unity iOS app to also send candidates of type srflx and relay because I had specified a STUN and TURN server. I verified the TURN server is working on TrickleICE, and the Web App is pointed at the same TURN server, and it is successful in gathering a relay candidate.

For some reason, I do not see the Unity iOS app sending any non-host ICE Servers. Do you have any insight as to what could be causing this issue?

[karasusan]

@eaberman This is an interesting result! I'll checking this later.

[kannan-xiao4]

memo: WRS-468

[eaberman]

Thank you @karasusan! I appreciate it!

[eaberman]

Hi @karasusan, I wanted to add an update to this. I believe that I fixed the issue I was having.

I was specifying my TURN and STUN servers in Render Streaming Project Settings, and marked in the Signaling Manager object attached to my camera to use the default settings in project settings.

But, in the _Run function in Signaling Manager, this line of code returned an empty array:

RTCIceServer[] iceServers = settings.iceServers.OfType<RTCIceServer>().ToArray();

I found that the settings.iceServers array in my project was actually of type IceServer instead of RTCIceServer, so I fixed it by adding code to convert each IceServer to an RTCIceServer and construct an RTCIceServer[] iceServers array from them, and then it appeared to work!

Also, I replaced the line _conf = conf.GetValueOrDefault(new RTCConfiguration { iceServers = iceServers }); in the _Run function with simply _conf = new RTCConfiguration { iceServers = iceServers }; to ensure the new array was being used.

This seemed to fix my problem. Thank you for responding to my comment on this issue!

[karasusan]

Related issue #895

[karasusan]

We released a new version that the issue is fixed. This ticket will be closed if no response by the end this week.