[BUG]: After upgrading to 3.1.0 exp7, the 4K video stream is too laggy

[ViKeei]

Package version

3.1.0-exp.7

Environment

* OS:Windows 10
* Unity version:Unity2022.3.1
* Graphics API:DirectX11
* Browser:Microsoft Edge 116

Steps To Reproduce

After upgrading to 3.1.0 exp7, the 4K video stream is too laggy, the screen of 3.1.0 exp6 is very smooth. with the same scene : 3.1.0 exp7 : 4K 25FPS H264 3.1.0 exp6 : 4K 37FPS H264 Another issue is that Unity crashes when switching Unity Render Streaming versions and immediately connect to web video streaming.

Current Behavior

No response

Expected Behavior

No response

Anything else?

Video_20230818110332_edit.zip

[karasusan]

Hi, I think this fix is related your issue. We already merged it. https://github.com/Unity-Technologies/com.unity.webrtc/pull/962

[karasusan]

And as a workaround, the crush might not be occurred using DX12.

[ViKeei]

Things seem to be getting worse, but the 3.1.0 exp6 version is still fine. If opened in default mode, unity will crash directly.There are two types of crashes in the video. Video_20230818143031.zip

[karasusan]

@ViKeei Let me check this issue. Please tell me the graphics driver version just in case. Also, is this crash relating the 4K resolution? How about FHD resolution?

[ViKeei]

NVIDIA RTX 3060Ti, Game Ready 531.18 QHD and 4K UHD both crash, FHD not.

[karasusan]

@ViKeei Thanks, we're going to check the 4K resolution again.

[karasusan]

@ViKeei Sorry, I have a question, is this issue reproduced when using DX12 as a graphics device instead of DX11?

[ViKeei]

@karasusan At your suggestion, subsequent tests and replies have been done in DX12. Compared to exp6's version, exp7's version running under DX11 only results in 4K frame rate drops and choppy graphics.

[karasusan]

WRS-495

[karasusan]

@ViKeei Thanks, I want to know this fix is related your issue or not. Can you try it if I provide a package which contains this fix? https://github.com/Unity-Technologies/com.unity.webrtc/pull/962

[ViKeei]

@karasusan OK, no problem.

[karasusan]

@ViKeei https://drive.google.com/file/d/19COIA7sdb5jSd3Rijt9VDVy91BKyXZrQ/view?usp=drive_link

[ViKeei]

I tested it on both of my computers and both crashed. One is Windows10, one is Windows11.Test in Unity2022.3.1，Unity2023.1.8. In Unity2022, it crashes the first time it runs, and after restarting, the web page screen appears but the screen doesn't move and the bitrate is zero. In Unity2023, it crashes every time.

[karasusan]

@ViKeei OK, this crash must be a new issue, I put a zip file contains the native plugin built for debug. Please try it and send me a crash log. https://drive.google.com/file/d/1MqZtJgUCeRNEXm4MbgJaneEfm-OoMTWV/view?usp=drive_link

[ViKeei]

@karasusan Editor.log

[ViKeei]

this is from Unity 2023.1.8 Crashes.zip

[karasusan]

@ViKeei I am trying to replicate your issue on my windows machine, but I couldn't yet. The assertion means that ID3D12Resource::Map is failed in D3D12GraphicsDevice.cpp.

I don't think there are differences between Unity Editor and one of China version.

[ViKeei]

@karasusan I can't say for sure if it's an issue with the Chinese version, but I'm experiencing crashes in the non-Chinese version as well. I created multiple brand new projects, but the crashes didn't happen in every project, and in some of them I did a Unity render streaming version switch. But for sure the exp6 version works fine in all tests. Maybe it's just me.

Framerate comparison for the same scene:

This is a crash from Unity2023.1.8f1： Editor.log

[karasusan]

@ViKeei Thank you for the clarification. Is it possible to identify the difference of the crash project and the not crash? I would be good if I could check the minimum project for replicate the crash.

[lijunhe1991]

Is there a solution to this problem? I have the same problem here.

[karasusan]

@lijunhe1991 The performance issue with DX11 is replicated on your environment? I'd like to know more details like PC specs and driver version.

[lijunhe1991]

@karasusan

• OS:Windows 10（NVIDIA GeForce RTX 3060）

• Unity version:Unity2021.3.6f1

• Graphics API:DirectX11

• Browser:Google Chrome 116.0.5845.112

The default frame rate is 30 frames, the 1K frame rate is normal, and the 4K frame rate is only about 10 frames and unstable.

At 2023-09-01 10:08:58, "Kazuki Matsumoto" @.***> wrote:

@lijunhe1991 The performance issue with DX11 is replicated on your environment? I'd like to know more details like PC specs and driver version.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[lijunhe1991]

@karasusan Is there a solution to this problem?

[karasusan]

@lijunhe1991 @ViKeei

It looks no problem in my environment.

https://github.com/Unity-Technologies/UnityRenderStreaming/assets/1132081/5f297c7c-afc0-4e6b-87e6-96c493fece63

This is my system info.

NVIDIA System Information report created on: 09/15/2023 18:05:50 System name: LAPTOP-HBGEGGU7

[Display] Operating System: Windows 10 Pro, 64-bit DirectX version: 12.0 GPU processor: Quadro RTX 5000 Driver version: 529.08 Driver Type: DCH Direct3D feature level: 12_1 CUDA Cores: 3072 Core clock: 1350 MHz Memory data rate: 12.00 Gbps Memory interface: 256-bit Memory bandwidth: 384.06 GB/s Total available graphics memory: 32619 MB Dedicated video memory: 16384 MB GDDR6 System video memory: 0 MB Shared system memory: 16235 MB Video BIOS version: 90.04.7D.00.17 IRQ: Not used Bus: PCI Express x16 Gen3 Device Id: 10DE 1EB5 229717AA Part Number: 4914 0010

[Components]

nvui.dll 8.17.15.2908 NVIDIA User Experience Driver Component nvxdplcy.dll 8.17.15.2908 NVIDIA User Experience Driver Component nvxdbat.dll 8.17.15.2908 NVIDIA User Experience Driver Component nvxdapix.dll 8.17.15.2908 NVIDIA User Experience Driver Component NVCPL.DLL 8.17.15.2908 NVIDIA User Experience Driver Component nvCplUIR.dll 8.1.940.0 NVIDIA Control Panel nvCplUI.exe 8.1.940.0 NVIDIA Control Panel nvWSSR.dll 31.0.15.2908 NVIDIA Workstation Server nvWSS.dll 31.0.15.2908 NVIDIA Workstation Server nvViTvSR.dll 31.0.15.2908 NVIDIA Video Server nvViTvS.dll 31.0.15.2908 NVIDIA Video Server nvLicensingS.dll 6.14.15.2908 NVIDIA Licensing Server nvDevToolSR.dll 31.0.15.2908 NVIDIA Licensing Server nvDevToolS.dll 31.0.15.2908 NVIDIA 3D Settings Server nvDispSR.dll 31.0.15.2908 NVIDIA Display Server nvDispS.dll 31.0.15.2908 NVIDIA Display Server PhysX 09.21.0713 NVIDIA PhysX NVCUDA64.DLL 31.0.15.2908 NVIDIA CUDA 12.0.151 driver nvGameSR.dll 31.0.15.2908 NVIDIA 3D Settings Server nvGameS.dll 31.0.15.2908 NVIDIA 3D Settings Server

[lijunhe1991]

@karasusan H264 frame rate is stable and normal，other codec frame rate is very low.

[karasusan]

@lijunhe199 Generally, encoding 4K video with software codec is verh high load. I recommend you to use H264 codec.

[lijunhe1991]

Is there any plan to support GPU encoding for other encoding formats?

[karasusan]

@lijunhe1991 I think AV1 NVIDIA codec should be supported. Also oneVPL codec support is useful for many customers. However, I can't promise when we support them.