Sunshine icon indicating copy to clipboard operation
Sunshine copied to clipboard

Published 20 hours ago • verified publisher icon LizardByte

Support native YUV 4:4:4 encoding (Windows-only for now)

Open ns6089 opened this issue 1 year ago • 23 comments

Description

Adds support for YUV 4:4:4 encoding, requires changes on moonlight side. Windows-only for now.

moonlight-common-c pull request: https://github.com/moonlight-stream/moonlight-common-c/pull/91 merged moonlight-qt pull request: https://github.com/moonlight-stream/moonlight-qt/pull/1282 merged

Current state

  • nvidia gpus support is implemented by using nvenc directly (Windows-only)
  • intel gpus support was implemented blindly, no idea if it works at all (Windows-only)
  • amd gpus don't support YUV 4:4:4 at all
    • it's possible to emulate 4:4:4 on 4:2:0 with custom shaders, working on this here https://github.com/LizardByte/Sunshine/pull/2760
  • nvenc doesn't accept any direct3d surfaces for 10-bit 4:4:4 encoding, so we have to use cuda interop
    • cuda runtime can't be unloaded once loaded
      • it doesn't seem to affect gpu idle power state, so we should be fine
    • ~~nvenc in cuda mode leaks cpu memory on decoder destruction (nvenc-mapped cuda surfaces can't be unmapped and unregistered)~~
      • fixed by slightly adjusting the api calls
  • linux support may be possible through ffmpeg, but not yet implemented or even investigated

Screenshot

moonlight_yuv444

Issues Fixed or Closed

Type of Change

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [ ] Dependency update (updates to dependencies)
  • [ ] Documentation update (changes to documentation)
  • [ ] Repository update (changes to repository files, e.g. .github/...)

Checklist

  • [x] My code follows the style guidelines of this project
  • [x] I have performed a self-review of my own code
  • [x] I have commented my code, particularly in hard-to-understand areas
  • [x] I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch must be updated before it can be merged. You must also Allow edits from maintainers.

  • [x] I want maintainers to keep my branch updated

ns6089 avatar May 15 '24 09:05 ns6089

Cool, didn't realize NVENC supported that (even though AV1 is still software only right?). And I see an image is worth a thousand words. Would there also be benefits with the lossless profile? Or is that just a bandwidth (and/or latency?) monstrosity?

mirh avatar May 28 '24 08:05 mirh

AV1 is still software only right?

Yes, no hardware encoding for AV1 4:4:4 on current generation of gpus

Would there also be benefits with the lossless profile?

I'm not against the idea of supporting lossless encoding options (NVENC can do it for H.264 and HEVC, not AV1), but current netcode imposes hard limit on maximum video packet size, and this limit is very easy to hit on lossless. So need to improve the netcode first.

ns6089 avatar May 28 '24 08:05 ns6089

Supposedly.. it may even be possible to dynamically switch between lossy and lossless? For as much as maybe it's not really necessary, if yuv444 can already score a SSIM of 0.98.

mirh avatar May 28 '24 11:05 mirh

Supposedly.. it may even be possible to dynamically switch between lossy and lossless?

You're describing near lossless encoding, or including both DCT transform and quantization bypasses into rate control assessment. As far as I know, NVENC is not capable of this (quantization can be dynamically lossless when rate control selects QP=4, but DCT transform bypass is static on/off switch).

ns6089 avatar May 28 '24 12:05 ns6089

Should be more or less done. Still need to figure out how to best handle the NVENC/CUDA driver bug, and maybe adjust how 8-bit colors are mapped into 10-bit colors depending on particular client renderer (255 colors don't evenly map into 1023 colors). But the code should be already complete and correct, as far as I can tell.

ns6089 avatar Jun 25 '24 16:06 ns6089

Ah, and I still have no idea if Intel encoder works correctly since I don't have supported hardware at hand right now.

ns6089 avatar Jun 25 '24 16:06 ns6089

Resolved the CUDA bug, only minor stuff is left.

ns6089 avatar Jun 28 '24 11:06 ns6089

We have some new patterns for docs which help produce cleaner doxygen docs. https://docs.lizardbyte.dev/projects/sunshine/en/master/source_code/source_code.html

ReenigneArcher avatar Jun 28 '24 16:06 ReenigneArcher

Alright, I will update the comments to what the codebase will be using at the time of merge. Currently this PR is held back at moonlight side, and merging one without the other is pointless.

ns6089 avatar Jun 28 '24 18:06 ns6089

Currently this PR is held back at moonlight side

I will mark this as draft. Please mark it ready, once moonlight side is taken care of.

ReenigneArcher avatar Jul 14 '24 23:07 ReenigneArcher

Reopening since moonlight side is ready to be merged. I've also updated doxygen comments to follow the rest of the codebase.

ns6089 avatar Jul 17 '24 06:07 ns6089

Thank you for your great work. I just built both moonlight and sunshine but faced green screen.

What did i wrong?

[2024:07:17:19:58:16]: Info: Sunshine version: 0.0.0.02254959 [2024:07:17:19:58:16]: Info: nvprefs: No need to modify application profile settings [2024:07:17:19:58:16]: Info: nvprefs: Changed OGL_CPL_PREFER_DXPRESENT to OGL_CPL_PREFER_DXPRESENT_PREFER_ENABLED for base profile [2024:07:17:19:58:16]: Info: Compiling shaders... [2024:07:17:19:58:16]: Info: System tray created [2024:07:17:19:58:16]: Info: Compiled shaders [2024:07:17:19:58:16]: Warning: Gamepad x360 is disabled due to gamepads.vigem-not-available [2024:07:17:19:58:16]: Warning: Gamepad ds4 is disabled due to gamepads.vigem-not-available [2024:07:17:19:58:16]: Warning: No gamepad input is available [2024:07:17:19:58:16]: Info: Trying encoder [nvenc] [2024:07:17:19:58:16]: Info: ddprobe.exe [1] [] returned: 0x00000000 [2024:07:17:19:58:16]: Info: Set GPU preference: 1 [2024:07:17:19:58:16]: Info: Device Description : NVIDIA GeForce RTX 4060 Device Vendor ID : 0x000010DE Device Device ID : 0x00002882 Device Video Mem : 7957 MiB Device Sys Mem : 0 MiB Share Sys Mem : 16303 MiB Feature Level : 0x0000B100 Capture size : 1920x1080 Offset : 0x0 Virtual Desktop : 1920x1080 [2024:07:17:19:58:16]: Info: Active GPU has HAGS enabled [2024:07:17:19:58:16]: Info: Using realtime GPU priority [2024:07:17:19:58:16]: Info: Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709 Bits Per Color : 8 Red Primary : [0.639648,0.330078] Green Primary : [0.299805,0.599609] Blue Primary : [0.149414,0.0595703] White Point : [0.3125,0.329102] Min Luminance : 0.5 nits Max Luminance : 270 nits Max Full Luminance : 270 nits [2024:07:17:19:58:16]: Info: Desktop resolution [1920x1080] [2024:07:17:19:58:16]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM] [2024:07:17:19:58:16]: Info: Display refresh rate [144Hz] [2024:07:17:19:58:16]: Info: Requested frame rate [60fps] [2024:07:17:19:58:16]: Info: SDR color coding [Rec. 601] [2024:07:17:19:58:16]: Info: Color depth: 8-bit [2024:07:17:19:58:16]: Info: Color range: [JPEG] [2024:07:17:19:58:17]: Info: NvEnc: created encoder H.264 P1 two-pass rfi [2024:07:17:19:58:17]: Info: SDR color coding [Rec. 601] [2024:07:17:19:58:17]: Info: Color depth: 8-bit [2024:07:17:19:58:17]: Info: Color range: [JPEG] [2024:07:17:19:58:17]: Info: NvEnc: created encoder HEVC P1 two-pass rfi [2024:07:17:19:58:17]: Info: SDR color coding [Rec. 601] [2024:07:17:19:58:17]: Info: Color depth: 8-bit [2024:07:17:19:58:17]: Info: Color range: [JPEG] [2024:07:17:19:58:17]: Info: NvEnc: created encoder AV1 P1 two-pass rfi [2024:07:17:19:58:17]: Info: SDR color coding [Rec. 601] [2024:07:17:19:58:17]: Info: Color depth: 8-bit [2024:07:17:19:58:17]: Info: Color range: [JPEG] [2024:07:17:19:58:17]: Info: NvEnc: created encoder H.264 P1 yuv444 two-pass rfi [2024:07:17:19:58:17]: Info: Device Description : NVIDIA GeForce RTX 4060 Device Vendor ID : 0x000010DE Device Device ID : 0x00002882 Device Video Mem : 7957 MiB Device Sys Mem : 0 MiB Share Sys Mem : 16303 MiB Feature Level : 0x0000B100 Capture size : 1920x1080 Offset : 0x0 Virtual Desktop : 1920x1080 [2024:07:17:19:58:17]: Info: Active GPU has HAGS enabled [2024:07:17:19:58:17]: Info: Using realtime GPU priority [2024:07:17:19:58:17]: Info: Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709 Bits Per Color : 8 Red Primary : [0.639648,0.330078] Green Primary : [0.299805,0.599609] Blue Primary : [0.149414,0.0595703] White Point : [0.3125,0.329102] Min Luminance : 0.5 nits Max Luminance : 270 nits Max Full Luminance : 270 nits [2024:07:17:19:58:17]: Info: Desktop resolution [1920x1080] [2024:07:17:19:58:17]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM] [2024:07:17:19:58:17]: Info: Display refresh rate [144Hz] [2024:07:17:19:58:17]: Info: Requested frame rate [60fps] [2024:07:17:19:58:17]: Info: SDR color coding [Rec. 709] [2024:07:17:19:58:17]: Info: Color depth: 10-bit [2024:07:17:19:58:17]: Info: Color range: [JPEG] [2024:07:17:19:58:17]: Info: NvEnc: created encoder HEVC P1 yuv444 10-bit two-pass rfi [2024:07:17:19:58:17]: Info: SDR color coding [Rec. 709] [2024:07:17:19:58:17]: Info: Color depth: 10-bit [2024:07:17:19:58:17]: Info: Color range: [JPEG] [2024:07:17:19:58:17]: Error: NvEnc: gpu doesn't support YUV444 encode [2024:07:17:19:58:17]: Info: SDR color coding [Rec. 709] [2024:07:17:19:58:17]: Info: Color depth: 10-bit [2024:07:17:19:58:17]: Info: Color range: [JPEG] [2024:07:17:19:58:17]: Info: NvEnc: created encoder AV1 P1 10-bit two-pass rfi [2024:07:17:19:58:17]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. // [2024:07:17:19:58:17]: Info: [2024:07:17:19:58:17]: Info: // Ignore any errors mentioned above, they are not relevant. // [2024:07:17:19:58:17]: Info: [2024:07:17:19:58:17]: Info: Found H.264 encoder: h264_nvenc [nvenc] [2024:07:17:19:58:17]: Info: Found HEVC encoder: hevc_nvenc [nvenc] [2024:07:17:19:58:17]: Info: Found AV1 encoder: av1_nvenc [nvenc] [2024:07:17:19:58:17]: Info: Configuration UI available at [https://localhost:47990] [2024:07:17:19:58:18]: Info: Registered Sunshine mDNS service [2024:07:17:19:58:50]: Info: Encoder reenumeration is required [2024:07:17:19:58:50]: Info: Trying encoder [nvenc] [2024:07:17:19:58:50]: Info: Device Description : NVIDIA GeForce RTX 4060 Device Vendor ID : 0x000010DE Device Device ID : 0x00002882 Device Video Mem : 7957 MiB Device Sys Mem : 0 MiB Share Sys Mem : 16303 MiB Feature Level : 0x0000B100 Capture size : 1920x1080 Offset : 0x0 Virtual Desktop : 1920x1080 [2024:07:17:19:58:50]: Info: Active GPU has HAGS enabled [2024:07:17:19:58:50]: Info: Using realtime GPU priority [2024:07:17:19:58:50]: Info: Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709 Bits Per Color : 8 Red Primary : [0.639648,0.330078] Green Primary : [0.299805,0.599609] Blue Primary : [0.149414,0.0595703] White Point : [0.3125,0.329102] Min Luminance : 0.5 nits Max Luminance : 270 nits Max Full Luminance : 270 nits [2024:07:17:19:58:50]: Info: Desktop resolution [1920x1080] [2024:07:17:19:58:50]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM] [2024:07:17:19:58:50]: Info: Display refresh rate [144Hz] [2024:07:17:19:58:50]: Info: Requested frame rate [60fps] [2024:07:17:19:58:50]: Info: SDR color coding [Rec. 601] [2024:07:17:19:58:50]: Info: Color depth: 8-bit [2024:07:17:19:58:50]: Info: Color range: [JPEG] [2024:07:17:19:58:50]: Info: NvEnc: created encoder H.264 P1 two-pass rfi [2024:07:17:19:58:50]: Info: SDR color coding [Rec. 601] [2024:07:17:19:58:50]: Info: Color depth: 8-bit [2024:07:17:19:58:50]: Info: Color range: [JPEG] [2024:07:17:19:58:50]: Info: NvEnc: created encoder HEVC P1 two-pass rfi [2024:07:17:19:58:50]: Info: SDR color coding [Rec. 601] [2024:07:17:19:58:50]: Info: Color depth: 8-bit [2024:07:17:19:58:50]: Info: Color range: [JPEG] [2024:07:17:19:58:50]: Info: NvEnc: created encoder AV1 P1 two-pass rfi [2024:07:17:19:58:50]: Info: SDR color coding [Rec. 601] [2024:07:17:19:58:50]: Info: Color depth: 8-bit [2024:07:17:19:58:50]: Info: Color range: [JPEG] [2024:07:17:19:58:50]: Info: NvEnc: created encoder H.264 P1 yuv444 two-pass rfi [2024:07:17:19:58:50]: Info: Device Description : NVIDIA GeForce RTX 4060 Device Vendor ID : 0x000010DE Device Device ID : 0x00002882 Device Video Mem : 7957 MiB Device Sys Mem : 0 MiB Share Sys Mem : 16303 MiB Feature Level : 0x0000B100 Capture size : 1920x1080 Offset : 0x0 Virtual Desktop : 1920x1080 [2024:07:17:19:58:50]: Info: Active GPU has HAGS enabled [2024:07:17:19:58:50]: Info: Using realtime GPU priority [2024:07:17:19:58:50]: Info: Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709 Bits Per Color : 8 Red Primary : [0.639648,0.330078] Green Primary : [0.299805,0.599609] Blue Primary : [0.149414,0.0595703] White Point : [0.3125,0.329102] Min Luminance : 0.5 nits Max Luminance : 270 nits Max Full Luminance : 270 nits [2024:07:17:19:58:50]: Info: Desktop resolution [1920x1080] [2024:07:17:19:58:50]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM] [2024:07:17:19:58:50]: Info: Display refresh rate [144Hz] [2024:07:17:19:58:50]: Info: Requested frame rate [60fps] [2024:07:17:19:58:50]: Info: SDR color coding [Rec. 709] [2024:07:17:19:58:50]: Info: Color depth: 10-bit [2024:07:17:19:58:50]: Info: Color range: [JPEG] [2024:07:17:19:58:50]: Info: NvEnc: created encoder HEVC P1 yuv444 10-bit two-pass rfi [2024:07:17:19:58:50]: Info: SDR color coding [Rec. 709] [2024:07:17:19:58:50]: Info: Color depth: 10-bit [2024:07:17:19:58:50]: Info: Color range: [JPEG] [2024:07:17:19:58:50]: Error: NvEnc: gpu doesn't support YUV444 encode [2024:07:17:19:58:50]: Info: SDR color coding [Rec. 709] [2024:07:17:19:58:50]: Info: Color depth: 10-bit [2024:07:17:19:58:50]: Info: Color range: [JPEG] [2024:07:17:19:58:50]: Info: NvEnc: created encoder AV1 P1 10-bit two-pass rfi [2024:07:17:19:58:50]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. // [2024:07:17:19:58:50]: Info: [2024:07:17:19:58:50]: Info: // Ignore any errors mentioned above, they are not relevant. // [2024:07:17:19:58:50]: Info: [2024:07:17:19:58:50]: Info: Found H.264 encoder: h264_nvenc [nvenc] [2024:07:17:19:58:50]: Info: Found HEVC encoder: hevc_nvenc [nvenc] [2024:07:17:19:58:50]: Info: Found AV1 encoder: av1_nvenc [nvenc] [2024:07:17:19:58:50]: Info: Executing [Desktop] [2024:07:17:19:58:51]: Info: nvprefs: No need to modify application profile settings [2024:07:17:19:58:51]: Info: nvprefs: No need to modify global profile settings [2024:07:17:19:58:51]: Info: CLIENT CONNECTED [2024:07:17:19:58:51]: Info: Device Description : NVIDIA GeForce RTX 4060 Device Vendor ID : 0x000010DE Device Device ID : 0x00002882 Device Video Mem : 7957 MiB Device Sys Mem : 0 MiB Share Sys Mem : 16303 MiB Feature Level : 0x0000B100 Capture size : 1920x1080 Offset : 0x0 Virtual Desktop : 1920x1080 [2024:07:17:19:58:51]: Info: Active GPU has HAGS enabled [2024:07:17:19:58:51]: Info: Using realtime GPU priority [2024:07:17:19:58:51]: Info: Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709 Bits Per Color : 8 Red Primary : [0.639648,0.330078] Green Primary : [0.299805,0.599609] Blue Primary : [0.149414,0.0595703] White Point : [0.3125,0.329102] Min Luminance : 0.5 nits Max Luminance : 270 nits Max Full Luminance : 270 nits [2024:07:17:19:58:51]: Info: Desktop resolution [1920x1080] [2024:07:17:19:58:51]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM] [2024:07:17:19:58:51]: Info: Display refresh rate [144Hz] [2024:07:17:19:58:51]: Info: Requested frame rate [60fps] [2024:07:17:19:58:51]: Info: Capture format [DXGI_FORMAT_B8G8R8A8_UNORM] [2024:07:17:19:58:51]: Info: SDR color coding [Rec. 709] [2024:07:17:19:58:51]: Info: Color depth: 8-bit [2024:07:17:19:58:51]: Info: Color range: [JPEG] [2024:07:17:19:58:52]: Info: NvEnc: created encoder HEVC P1 yuv444 two-pass rfi [2024:07:17:19:58:52]: Info: Opus initialized: 48 kHz, 2 channels, 512 kbps (total), LOWDELAY [2024:07:17:19:58:58]: Info: CLIENT DISCONNECTED

Kobain-Seo avatar Jul 17 '24 11:07 Kobain-Seo

I just built both moonlight and sunshine but faced green screen. What did i wrong?

That's a very good question, I don't think I broke anything during rebase because it still works for me.

Ideally someone should git the CI button so build artifacts can be distributed,

ns6089 avatar Jul 17 '24 12:07 ns6089

Codecov Report

Attention: Patch coverage is 8.20896% with 492 lines in your changes missing coverage. Please review.

Project coverage is 9.67%. Comparing base (e8c837f) to head (fd58f6b). Report is 1 commits behind head on master.

Files Patch % Lines
src/platform/windows/display_vram.cpp 0.00% 163 Missing and 10 partials :warning:
src/nvenc/nvenc_d3d11_on_cuda.cpp 0.00% 151 Missing :warning:
src/nvenc/nvenc_base.cpp 0.00% 19 Missing and 23 partials :warning:
src/nvenc/nvenc_d3d11_native.cpp 0.00% 36 Missing :warning:
src/video.cpp 50.68% 22 Missing and 14 partials :warning:
src/video_colorspace.cpp 0.00% 17 Missing and 1 partial :warning:
src/nvhttp.cpp 0.00% 5 Missing and 5 partials :warning:
src/nvenc/nvenc_utils.cpp 0.00% 6 Missing :warning:
src/nvenc/nvenc_d3d11_on_cuda.h 0.00% 5 Missing :warning:
src/nvenc/nvenc_d3d11.cpp 0.00% 4 Missing :warning:
... and 6 more
Additional details and impacted files 
@@            Coverage Diff            @@
##           master   #2533      +/-   ##
=========================================
- Coverage    9.69%   9.67%   -0.02%     
=========================================
  Files          97     101       +4     
  Lines       17525   17898     +373     
  Branches     8204    8365     +161     
=========================================
+ Hits         1699    1732      +33     
- Misses      12988   13295     +307     
- Partials     2838    2871      +33
Flag Coverage Δ
Linux 7.28% <15.44%> (+0.08%) :arrow_up:
Windows 5.07% <2.13%> (-0.05%) :arrow_down:
macOS-12 10.49% <20.40%> (+0.12%) :arrow_up:
macOS-13 10.40% <20.40%> (+0.13%) :arrow_up:
macOS-14 10.72% <20.40%> (+0.13%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
src/nvenc/nvenc_encoded_frame.h 0.00% <ø> (ø)
src/nvenc/nvenc_base.h 0.00% <0.00%> (ø)
src/video.h 55.69% <87.50%> (+2.91%) :arrow_up:
src/nvenc/nvenc_d3d11.h 0.00% <0.00%> (ø)
src/rtsp.cpp 1.88% <0.00%> (-0.01%) :arrow_down:
src/utility.h 28.24% <0.00%> (-0.27%) :arrow_down:
src/platform/common.h 33.33% <0.00%> (-1.12%) :arrow_down:
src/nvenc/nvenc_d3d11.cpp 0.00% <0.00%> (-3.85%) :arrow_down:
src/nvenc/nvenc_d3d11_on_cuda.h 0.00% <0.00%> (ø)
src/nvenc/nvenc_utils.cpp 0.00% <0.00%> (ø)
... and 7 more

... and 1 file with indirect coverage changes

codecov[bot] avatar Jul 17 '24 13:07 codecov[bot]

Thank you for your great work. I just built both moonlight and sunshine but faced green screen.

CI build for Sunshine is ready now. Assuming you have a Windows host: https://github.com/LizardByte/Sunshine/actions/runs/9969085431/artifacts/1710861456

KWottrich avatar Jul 17 '24 13:07 KWottrich

I have the same green screen issue than @Kobain-Seo above, with a RTX A2000 as a host and an Intel Xe (13th Gen) as a client. However I didn't built it myself, I used those artifacts:

I'm very interested by this feature, so I can help on host/client testing on Windows 10/11 with RTX A2000, Intel Xe (i7-1365U) and Intel UHD 620 (i7-8665U)

Chouffy avatar Jul 18 '24 07:07 Chouffy

@Chouffy have you tried software decoding? It worked for me, so it is not sunshine issue. It's moonlight igpu decoding issue.

Kobain-Seo avatar Jul 18 '24 07:07 Kobain-Seo

It works great with the Geforce RTX 3060 Ti and a Windows 11 host. However, 4:4:4 encoding does not work with the Intel Core i5 11320H (Tiger Lake, Iris Xe) and a Windows 11 host combination. It works fine with Parsec Warp. Additionally, the logs indicate YUV444: supported .

CleanShot_2024-08-02_06-25-48

[2024:08:02:05:51:50]: Debug: ------  h264 ------
[2024:08:02:05:51:50]: Debug: PASSED: supported
[2024:08:02:05:51:50]: Debug: REF_FRAMES_RESTRICT: supported
[2024:08:02:05:51:50]: Debug: CBR: supported
[2024:08:02:05:51:50]: Debug: DYNAMIC_RANGE: unsupported
[2024:08:02:05:51:50]: Debug: YUV444: supported
[2024:08:02:05:51:50]: Debug: VUI_PARAMETERS: supported
[2024:08:02:05:51:50]: Debug: -------------------
[2024:08:02:05:51:50]: Info: Found H.264 encoder: h264_qsv [quicksync]
[2024:08:02:05:51:50]: Debug: ------  hevc ------
[2024:08:02:05:51:50]: Debug: PASSED: supported
[2024:08:02:05:51:50]: Debug: REF_FRAMES_RESTRICT: supported
[2024:08:02:05:51:50]: Debug: CBR: supported
[2024:08:02:05:51:50]: Debug: DYNAMIC_RANGE: supported
[2024:08:02:05:51:50]: Debug: YUV444: supported
[2024:08:02:05:51:50]: Debug: VUI_PARAMETERS: supported
[2024:08:02:05:51:50]: Debug: -------------------
[2024:08:02:05:51:50]: Info: Found HEVC encoder: hevc_qsv [quicksync]

I am using the macOS client. Moonlight and Sunshine use the following builds:

  • Moonlight: https://ci.appveyor.com/project/cgutman/moonlight-qt/builds/50287816/job/p870yhl7y1gd587n/artifacts
  • Sunshine: https://github.com/LizardByte/Sunshine/actions/runs/9998167287?pr=2533 (0.0.0.cf6c78dbe432e1783566b45ddf3f4b9a03a161dc)

sunshine_verbose_log.txt

mohemohe avatar Aug 01 '24 21:08 mohemohe

@mohemohe Much appreciated, there was a redundant check that I missed. Pushed the fix, new CI builds should be ready shortly.

ns6089 avatar Aug 01 '24 22:08 ns6089

@ns6089 Thanks for the quick response. I can now connect with YUV 4:4:4 enabled, but it does not seem to be encoded in 4:4:4.

There is no longer the hassle of switching options for each host.

Moonlight: https://ci.appveyor.com/project/cgutman/moonlight-qt/builds/50287816/job/p870yhl7y1gd587n/artifacts Sunshine: https://github.com/LizardByte/Sunshine/actions/runs/10206571176 (0.0.0.cd06345e30d2569961370781cbf64aa374d1c900)

CleanShot_2024-08-02_11-18-23

sunshine_verbose_log_cd06345e30d2569961370781cbf64aa374d1c900.txt

mohemohe avatar Aug 02 '24 03:08 mohemohe

Hmm, it appears we send the right surfaces but then ffmpeg misconfigures the encoder and internal conversion happens. This one is going to be tougher....

ns6089 avatar Aug 02 '24 08:08 ns6089

@mohemohe I tried to nudge quicksync in the right direction, CI builds are up. If it doesn't work we will have to patch ffmpeg 😞

ns6089 avatar Aug 02 '24 12:08 ns6089

@ns6089 I tried build #8648 and it appears to be encoding correctly in YUV4:4:4. Thank you for your great work. CleanShot_2024-08-03_17-45-30

mohemohe avatar Aug 03 '24 08:08 mohemohe

That's a bummer, but in this case I'm not sure what we can even do with the degree of QSV functionality that ffmpeg exposes. What we need is MFXVideoENCODE_Query()

If the in parameter is non-zero, the function checks the validity of the fields in the input structure. Then the function returns the corrected values in the output structure. If there is insufficient information to determine the validity or correction is impossible, the function zeroes the fields. This feature can verify whether the implementation supports certain profiles, levels or bitrates.

ns6089 avatar Aug 16 '24 06:08 ns6089

I guess we can simply blacklist h.264 and av1 4:4:4 for qsv bypassing ffmpeg entirely, should be enough for a while.

ns6089 avatar Aug 16 '24 07:08 ns6089

Either way, let's go ahead and merge this in its current state. I will add h.264/av1 blacklist shortly after.

ns6089 avatar Aug 16 '24 08:08 ns6089

@cgutman Something like this https://github.com/LizardByte/Sunshine/pull/3029 https://github.com/LizardByte/Sunshine/pull/3029/commits/6488a2c323dcd11b13f25a8dabb71dbe0d07fc04

ns6089 avatar Aug 16 '24 13:08 ns6089

Quality Gate Passed Quality Gate passed

Issues
47 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
1.6% Duplication on New Code

See analysis details on SonarCloud

sonarqubecloud[bot] avatar Aug 16 '24 15:08 sonarqubecloud[bot]