[BUG]: SendOnly Unity to RecvOnly browser makes Hardware Encoder fall back to VP8

[yvanzine]

Package version

Others

Environment

* OS: Ubuntu 20.04
* Unity version: 2022.1.3f1
* Graphics API: Vulkan
* Browser: Chrome
* Using DEVELOP branch for both webrtc and UnityRenderStreaming packages (feat/stability-project-develop)

Steps To Reproduce

There is motivation to use Sendonly on Unity peer and Recvonly on web browser because bitrate may be affected by upload speed of user and the fact that the bandwidth is asymmetric, download speed is great and upload speed is poor.

Sendonly was removed recently from the AddSenderTrack: public RTCRtpTransceiver AddSenderTrack(string connectionId, MediaStreamTrack track) { var peer = _mapConnectionIdAndPeer[connectionId]; RTCRtpSender sender = peer.peer.AddTrack(track); var transceiver = peer.peer.GetTransceivers().First(t => t.Sender == sender); // note:: This line is needed to stream video to other peers with hardware codec. // The exchanging SDP is failed if remove the line because the hardware decoder currently is not supported. // Please remove the line after supporting the hardware decoder. ~~transceiver.Direction = RTCRtpTransceiverDirection.SendOnly;~~ return transceiver; }

When I add the SendOnly back into the AddSenderTrack, H264 Hardware encoder is not offered to the web browser client and VP8 is instead provided.

Without this SendOnly line, I still see the SDP answer from browser come back with a=recvonly and I believe the video stream (NVENC H264 in Unity and ffmpeg on client ) is in fact sendrecv(unity)-recvonly(browser) but I would like to prove it. When I degrade upload bandwidth at the browser, the connection remains good.

Please confirm if this is the intended behavior of the change and close the ticket, if the connection is in fact sendonly-recvonly for video stream as intended.

Current Behavior

No response

Expected Behavior

No response

Anything else?

No response

[karasusan]

OK, I will check the issue. Thanks.

[ZeoWorks]

Can you open OBS and confirm that NVENC is an option for recording? Sometimes on Linux nvenc doesn't work right out of the box and needs specific nvidia drivers.

[yvanzine]

NVENC is a valid option, as I get H264 hardware encoding when sendrecv profile is used. I never use OSB for debugging webrtc or unity webrtc. Is it useful?

[karasusan]

@ZeoWorks Oh, could you tell me the detail of specific nvidia drivers?

[ZeoWorks]

When you have a fresh install of Linux with a nvidia GPU, they will use open source drivers (xserver), this does not have nvenc support built in. You need to manually install nvidia drivers such as 470, 510 or 515. Hope this helps.

[karasusan]

@ZeoWorks Definitely. To use NvCodec, we need to install the proprietary driver provided by NVIDIA.

[karasusan]

memo: URS-469

[kannan-xiao4]

@yvanzine

1. In the current develop, if you specify the H264 codec in the Receiver on the Browser side, Fallback does not occur and H264 is used. Try specifying the Sender codec in UnitySide once the #726 has been merged into Develop branch.
2. Changing to SendRecv is the default behavior of PeerConnection.AddTrack, so as intended. On the Broadcast sample in current develop branch, received Answer with RecvOnly on the Browser side, RTPRtcTransceiver.CurrentDirection of the Sender on the Unity side will be SendOnly.
3. Bitrate can only be changed on the Sender side, can't on the Receiver side. You can change Bitrate by using SetBitrateMethod.(We are currently creating a Bitrate change sample PR https://github.com/Unity-Technologies/UnityRenderStreaming/pull/730)

[kannan-xiao4]

@yvanzine Do you have any updates ? We close the issue if nothing.