feat(win): new capture method, Windows.Graphics.Capture

[tez011]

Description

This adds a capture backend on Windows that uses the Windows.Graphics.Capture API. This is a more recent API that's capable of capturing the Xbox Game Bar, which the DXGI Desktop Duplication API seemingly cannot do.

This pull request depends on C++20 compiler support (for the Microsoft WinRT headers, which define this API) as well as Universal CRT runtime support. These have already been integrated into Sunshine in #2322 and #2323.

See also #2149.

Issues Fixed or Closed

Fixes #832 (manually verified)

Screenshot

Issues Fixed or Closed

Type of Change

• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [ ] Dependency update (updates to dependencies)
• [x] Documentation update (changes to documentation)
• [x] Repository update (changes to repository files, e.g. .github/...)

Checklist

• [x] My code follows the style guidelines of this project
• [x] I have performed a self-review of my own code
• [x] I have commented my code, particularly in hard-to-understand areas
• [x] I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch must be updated before it can be merged. You must also Allow edits from maintainers.

• [x] I want maintainers to keep my branch updated

[tez011]

Right now, this pull request does not contain changes to support Sunshine's service model. Linking https://github.com/LizardByte/Sunshine/pull/2149#issuecomment-1955746301 for context.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 7.39300% with 238 lines in your changes missing coverage. Please review.

Project coverage is 6.86%. Comparing base (2d706d3) to head (f77eb15). Report is 153 commits behind head on master.

Files	Patch %	Lines
src/platform/windows/display_wgc.cpp	4.63%	138 Missing and 6 partials :warning:
src/platform/windows/display_base.cpp	12.69%	47 Missing and 8 partials :warning:
src/platform/windows/display_vram.cpp	7.89%	33 Missing and 2 partials :warning:
src/platform/windows/display_ram.cpp	20.00%	3 Missing and 1 partial :warning:

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #2580      +/-   ##
=========================================
- Coverage    7.03%   6.86%   -0.17%     
=========================================
  Files          87      88       +1     
  Lines       17698   17899     +201     
  Branches     8407    8514     +107     
=========================================
- Hits         1245    1229      -16     
- Misses      13717   14012     +295     
+ Partials     2736    2658      -78     

Flag	Coverage Δ
Linux	5.35% <ø> (ø)
Windows	2.62% <7.39%> (+0.06%)	:arrow_up:
macOS-12	8.05% <ø> (ø)
macOS-13	7.98% <ø> (+0.01%)	:arrow_up:
macOS-14	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
src/platform/windows/display.h	7.14% <ø> (-2.58%)	:arrow_down:
src/platform/windows/display_ram.cpp	0.99% <20.00%> (-0.02%)	:arrow_down:
src/platform/windows/display_vram.cpp	1.87% <7.89%> (+0.28%)	:arrow_up:
src/platform/windows/display_base.cpp	12.50% <12.69%> (+0.62%)	:arrow_up:
src/platform/windows/display_wgc.cpp	4.63% <4.63%> (ø)

... and 29 files with indirect coverage changes

[ReenigneArcher]

I've quickly tested this locally.

• CPU usage is < 4% while streaming with either capture method.
• RAM usage is < 5mb while streaming with either capture method.
• Can't use the new capture method when running it as a service, which I suppose is expected for now.
• Can start Sunshine from start menu shortcut.

https://github.com/LizardByte/Sunshine/pull/2320

Was there anything else broken when we tried to merge this before?

[tez011]

I seem to remember a couple other things:

• Configuration UI comes up no slower than usual (was previously broken due to regex)
• HDR may still not work (I don't have hardware capable of testing that)

[ReenigneArcher]

@tez011

UI seems to be coming up quickly for me.

HDR may still not work (I don't have hardware capable of testing that)

This was just with the new capture method, correct? The old one is unaffected?

[tez011]

HDR may still not work (I don't have hardware capable of testing that)

This was just with the new capture method, correct? The old one is unaffected?

Yes. I could never reproduce it, but I recall this discussion from the old pull request. https://github.com/LizardByte/Sunshine/pull/2149#issuecomment-2027437793

[ReenigneArcher]

Alright, this will be merged soon. Thank you.

[Kobain-Seo]

When i enabled this option, sunshine fails to start.

Logs:

[2024-06-20 14:24:29.019908] [0x00002e0c] [info] Gamepad x360 is disabled due to gamepads.vigem-not-available [2024-06-20 14:24:29.021147] [0x00002e0c] [info] Gamepad ds4 is disabled due to gamepads.vigem-not-available [2024-06-20 14:24:29.022127] [0x00002e0c] [info] Gamepad x360 is disabled due to gamepads.vigem-not-available [2024-06-20 14:24:29.023114] [0x00002e0c] [info] Gamepad ds4 is disabled due to gamepads.vigem-not-available [capture] -- [wgc] [encoder] -- [nvenc] [qp] -- [0] [nvenc_twopass] -- [full_res] [fec_percentage] -- [0] [controller] -- [disabled] [native_pen_touch] -- [disabled] [nvenc_vbv_increase] -- [40] [2024-06-20 14:24:29.024813] [0x00002e0c] [info] Gamepad x360 is disabled due to gamepads.vigem-not-available [2024-06-20 14:24:29.025803] [0x00002e0c] [info] Gamepad ds4 is disabled due to gamepads.vigem-not-available [2024:06:20:14:24:29]: Info: Sunshine version: v2024.620.13611 [2024:06:20:14:24:29]: Info: nvprefs: Opened undo file from previous improper termination [2024:06:20:14:24:29]: Info: nvprefs: Restored OGL_CPL_PREFER_DXPRESENT for base profile [2024:06:20:14:24:29]: Info: nvprefs: Restored global profile settings from undo file - deleting the file [2024:06:20:14:24:29]: Info: nvprefs: No need to modify application profile settings [2024:06:20:14:24:29]: Info: nvprefs: Changed OGL_CPL_PREFER_DXPRESENT to OGL_CPL_PREFER_DXPRESENT_PREFER_ENABLED for base profile [2024:06:20:14:24:29]: Info: Compiling shaders... [2024:06:20:14:24:29]: Info: System tray created [2024:06:20:14:24:29]: Info: Compiled shaders [2024:06:20:14:24:29]: Warning: Gamepad x360 is disabled due to gamepads.vigem-not-available [2024:06:20:14:24:29]: Warning: Gamepad ds4 is disabled due to gamepads.vigem-not-available [2024:06:20:14:24:29]: Warning: No gamepad input is available [2024:06:20:14:24:29]: Info: Trying encoder [nvenc] [2024:06:20:14:24:29]: Info: ddprobe.exe [1] [] returned: 0x00000000 [2024:06:20:14:24:29]: Info: Set GPU preference: 1 [2024:06:20:14:24:29]: Info: Device Description : NVIDIA GeForce RTX 3060 Device Vendor ID : 0x000010DE Device Device ID : 0x00002504 Device Video Mem : 12115 MiB Device Sys Mem : 0 MiB Share Sys Mem : 8113 MiB Feature Level : 0x0000B100 Capture size : 1920x1080 Offset : 0x0 Virtual Desktop : 1920x1080 [2024:06:20:14:24:29]: Info: Active GPU has HAGS enabled [2024:06:20:14:24:29]: Info: Using realtime GPU priority [2024:06:20:14:24:29]: Info: Colorspace : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709 Bits Per Color : 8 Red Primary : [0.639648,0.330078] Green Primary : [0.299805,0.599609] Blue Primary : [0.149414,0.0595703] White Point : [0.3125,0.329102] Min Luminance : 0.5 nits Max Luminance : 270 nits Max Full Luminance : 270 nits

[ReenigneArcher]

You have to disable the service and start Sunshine manually.

[Kobain-Seo]

You have to disable the service and start Sunshine manually.

I think it is not relevant to running manually. I tested it on Windows 10 22H2 and sunshine failed to run, and It worked on Windows 11 22H2. Is it normal it does not work on Windows 10 ?

[tez011]

Yes, that’s expected. This API is quite new.

[Kobain-Seo]

after patched #2825, It works on Windows 10, but have to be run manually, not by service.

[ReenigneArcher]

after patched #2825, It works on Windows 10, but have to be run manually, not by service.

For now, that's expected.

[PieroPontra]

Latest version of Win11 but the new beta WinGraphicsCapture that I switched to manually in configs says it can't find any encoder or display attached. I already tried to manually enter names from dxci command but same results, any hints?

[WXZhao7]

I noticed from https://github.com/LizardByte/Sunshine/pull/2149#issuecomment-1955746301 that WGC should be able to capture the UAC secure desktop. I have tested several pre-release versions, including v2024.714.230159. Unfortunately, Moonlight lose control when the UAC window pops up. Is the capture of UAC prompts by WGC still incomplete, or is it not planned for implementation?

[roob0]

@PieroPontra You have to close the service and launch Sunshine from the exe, but so you can't stream the lockscreen. However, I created a simple solution for switching automatically from service to exe and from ddx wo wgc (and viceversa) here: https://github.com/LizardByte/Sunshine/pull/2149

[karasuhebi]

Is there any other benefits to this capture method?