[BUG] Framerate is choppy, artifacts, when upgraded from 3.1.1-preview to 3.1.0-exp.2.

[yvanzine]

Framerate is much more unstable in 3.1.0-exp.2 vs 3.1.1-preview. Smearing artifacts are more often present.

We definitely need Render Thread - Main Thread decoupling fix.

@karasusan I think you less frequently see quality issues because all of the tests are written with 720P resolution. Even if you go up to 1080P, more artifacts are visible, including this one.

Environment (please complete the following information):

• OS: Linux, Ubuntu 20.04
• Browser chrome
• Unity Version 2020.3.24f
• Package version 3.1.0-exp.2

[karasusan]

@yvanzine Is this the same issue? https://github.com/Unity-Technologies/UnityRenderStreaming/issues/446

There is no version 3.1.1-preview. What version are you testing?

[yvanzine]

@karasusan

I am testing 3.0.1-exp.2 of course.

I am sorry I meant to say 3.0.1-preview had better, smoother framerate than 3.0.1-exp.2 and there is about 5 fps lost in the new version ( regression )

The smearing bug you reference #446 is there because of low max bitrate, I am pretty sure. It is NOT the issue here. Here it is just degradation of fps in 3.1.0-exp.2 and it is less constant than in 3.0.1-preview.

[karasusan]

@yvanzine In old version, we replaced SDP to change bitrate. Is it possible the change is caused your issue? https://github.com/Unity-Technologies/UnityRenderStreaming/issues/450#issue-837112899

[yvanzine]

@karasusan

No this issue of choppy framerate is NOT related to being able to munge SDP. Here I am merely reporting a framerate instability going from 3.1.1-preview to 3.1.0-exp.2. I am currently NOT migrating to 3.1.0-exp.2 because the framerate has become unstable.

There is no problem with bitrate any more on my end. I am able to allow for bitrate to rise in any new version by munging SDP information like I describe here: https://github.com/Unity-Technologies/com.unity.webrtc/issues/552

[yvanzine]

@karasusan

I wanted to add to this ticket that in addition to choppy framerate and laggy interaction problem in 3.1.0-exp.2, there is a crash that did not happen before. I will have to work to reproduce the crash and add notes here. Once I get back to testing 3.1.0-exp.2.

[karasusan]

@yvanzine Thanks for your report. I expect you are using HW encoder on Linux. We know the current implementation has issues about HW encoding. As you can see this thread, we are going to replace the implementation about HW encoder integration to fix issues like this. https://forum.unity.com/threads/unity-render-streaming-status-update-q4-2021.1215801/

[yvanzine]

@karasusan Yes you are correct, we are running the NVidia encoder on Linux. We have no issues with the HW encoder on 3.0.1-preview. I upgraded to the 6.2 encoder following instructions on the forum and I think we are pretty happy with the encoder performance.

[karasusan]

@yvanzine We checked the issue and reproduced it. Sorry for waiting but we are working on it. https://github.com/Unity-Technologies/com.unity.webrtc/issues/552

[karasusan]

memo: WRS-155

[karasusan]

Fixed the issue in the WebRTC package. We need to upgrade dependencies in Unity Render Streaming package. https://github.com/Unity-Technologies/com.unity.webrtc/pull/721