Sunshine icon indicating copy to clipboard operation
Sunshine copied to clipboard
Published 1 week ago • verified publisher icon LizardByte

Add capture using WinRT Windows.Graphics.Capture API.

Open tez011 opened this issue 1 year ago • 25 comments

Description

This adds a capture backend on Windows that uses the Windows.Graphics.Capture API. This is a more recent API that's capable of capturing the Xbox Game Bar, which the DXGI Desktop Duplication API seemingly cannot do.

To accomplish this, I used a slightly different build environment known as MSYS UCRT64. I also bumped the compilation standard to C++20 so that the GNU C++ compiler can handle the coroutine definitions present in WinRT headers. The UCRT64 and MinGW64 environment appear to be binary-compatible, and Sunshine's dependencies translate smoothly across this boundary. The build documentation in this change reflects these modifications.

This change can be enabled by setting capture = wgc in sunshine.conf on Windows servers. When this setting is off, no functionality change should be observable. I've observed CPU and memory usage trends over a long streaming period and there don't appear to be any new leaks.

Issues Fixed or Closed

Fixes #832 (manually verified)

Type of Change

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [x] Dependency update (updates to dependencies)
  • [x] Documentation update (changes to documentation)
  • [ ] Repository update (changes to repository files, e.g. .github/...)

Checklist

  • [x] My code follows the style guidelines of this project
  • [x] I have performed a self-review of my own code
  • [x] I have commented my code, particularly in hard-to-understand areas
  • [x] I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch must be updated before it can be merged. You must also Allow edits from maintainers.

  • [x] I want maintainers to keep my branch updated

tez011 avatar Feb 18 '24 22:02 tez011

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Feb 18 '24 22:02 CLAassistant

It looks like you need to also update the GitHub Actions to follow use the new build steps (build_win section): https://github.com/LizardByte/Sunshine/blob/nightly/.github/workflows/CI.yml

cgutman avatar Feb 19 '24 00:02 cgutman

Does UAC elevation and service mode work as expected? I've read many microsoft sources that pretty much told that this capture approach is not compatible with unattended use cases

Source: https://github.com/robmikh/Win32CaptureSample/issues/24 and https://github.com/robmikh/Win32CaptureSample/issues/54

TheElixZammuto avatar Feb 19 '24 18:02 TheElixZammuto

Thanks for the quick reviews! I'll be addressing code and GitHub actions issue in, uh, a bit.

One other thing that WGC apparently provides is support for cross-adapter capture (capturing outputs attached to GPU 1 to a texture on GPU 2 for encoding).

@cgutman: I have not tried this scenario and while I did see this somewhere, I figured it was a little out of scope considering that, as far as I can tell, Sunshine is consumer software.

Does UAC elevation and service mode work as expected? I've read many microsoft sources that pretty much told that this capture approach is not compatible with unattended use cases

@TheElixZammuto: I think the headless scenario will work, but I need to test it after I make requested changes. Privilege elevation probably won't, but the existing API supports it. I designed this feature to be opt-in-only and to fall back if unsupported, so I hope that there's sufficient insurance for this.

In no way do I advocate replacing the existing display capture code with Windows.Graphics.Capture. I quite like the idea of it being a beta feature that can be enabled in sunshine.conf but not the user interface. I also hope its presence lowers the bar for more knowledgeable contributors to improve this capture backend.

tez011 avatar Feb 20 '24 00:02 tez011

Does UAC elevation and service mode work as expected? I've read many microsoft sources that pretty much told that this capture approach is not compatible with unattended use cases

WGC actually can capture the UAC secure desktop, even from an unelevated Sunshine.exe process, so it's actually better than DXGI DDA in that regard. It can even capture the login screen after locking the PC.

However, you are correct that it doesn't work with our service model, at least not without additional changes. WGC works using per-user services which are activated via COM. The specific user service that runs the capture backend is CaptureService which is implemented in %SystemRoot%\system32\CaptureService.dll. The use of a per-user service is probably why the 0x80070424 error is thrown from a service context, because the service indeed doesn't exist for LocalSystem's LUID (which is how per-user services are identified).

Fortunately, the fact that elevation doesn't seem to actually provide any benefit to WGC (unlike DDA) means we do have some options here to support WGC from a service context.

The first and simplest is to try impersonating using the user token from WTSQueryUserToken() (same as we do to start apps). We might just need this for our first activation of WGC (which is the call to winrt::GraphicsCaptureSession::IsSupported()) or we could also need it across other WGC API calls.

If impersonation doesn't work out, we'd need to use a WGC helper process that we would spawn into the user session and share textures with. This is definitely much more complex than the impersonation solution, but it may not be too horrible since we're already using shared textures between capture and encoding today.

I have not tried this scenario and while I did see this somewhere, I figured it was a little out of scope considering that, as far as I can tell, Sunshine is consumer software.

Cross-adapter encoding is good to have on hybrid graphics systems, which are quite typical these days (most CPUs include an iGPU and basically all laptops do). Cross-adapter encoding lets users encoding on the dGPU even when the display is physically connected to the iGPU. We have to use a bunch of hacks (ddprobe.exe manually setting our graphics preference) to work around the cross-adapter encoding limitation for DDA on hybrid graphics systems to even be able to stream at all.

In no way do I advocate replacing the existing display capture code with Windows.Graphics.Capture. I quite like the idea of it being a beta feature that can be enabled in sunshine.conf but not the user interface. I also hope its presence lowers the bar for more knowledgeable contributors to improve this capture backend.

I actually would like to move to WGC as the default on OSes where support is good (Win11/2022+). It has a ton of advantages over the DXGI solution, including:

  • Ability to capture the Game Bar
  • Fixed frame rate capture
  • Cross-adapter capture (untested but claimed by OBS folks)
  • Avoids all the mess of having to blend the cursor into the frame
  • Non-blocking API that doesn't hold critical locks for long periods of time like IDXGIOutputDuplication::AcquireNextFrame() does
  • Could potentially work around the Nvidia driver bug that causes hangs with high VRAM usage (the deadlock in the capture thread is on the keyed mutex inside IDXGIOutputDuplication::AcquireNextFrame())
  • Potential for much better frame pacing and lower latency by avoiding sleeps in our capture loop (we can instead look at the current time in on_frame_arrived() and decide to encode a frame or not based on how long it's been since the last frame)

Obviously it could still have massive Achilles heel that sinks it for our usecase, but it looks quite compelling on the surface.

In any case, none of this aspirational stuff matters for right now. I'm mainly just writing it down somewhere to not forget any of it. You should leave it as opt-in for this PR as you have it now and don't worry about the impersonation stuff.

cgutman avatar Feb 21 '24 02:02 cgutman

Looks like CI is failing now because of deprecated declarations in ViGEmBus and -Werror. Maintainers, what changes would you suggest here?

tez011 avatar Feb 25 '24 20:02 tez011

what changes would you suggest here?

Properly rebase the branch... don't update the vigemclient submodule as part of this PR.

ReenigneArcher avatar Feb 25 '24 22:02 ReenigneArcher

-Werror still hurting CI due to deprecated declaration in ViGEmBus

(my statement above is incorrect, actually)**

tez011 avatar Feb 27 '24 00:02 tez011

-Werror still hurting CI due to deprecated declaration in ViGEmBus

I'm not sure what it could be. Something specific to this PR, as we're building in other PRs without issue.

Maybe something with using c++20?

ReenigneArcher avatar Feb 27 '24 00:02 ReenigneArcher

I’m traveling this week, but when I’m back I can create a new branch with the exact same changes on top of latest nightly, hopefully CI cooperates a bit more after that.

On Feb 26, 2024, at 4:56 PM, ReenigneArcher @.***> wrote:



-Werror still hurting CI due to deprecated declaration in ViGEmBus

I'm not sure what it could be. Something specific to this PR, as we're building in other PRs without issue.

— Reply to this email directly, view it on GitHubhttps://github.com/LizardByte/Sunshine/pull/2149#issuecomment-1965599783, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFJCO5DL22HF6JW2ODL5YR3YVUVJXAVCNFSM6AAAAABDOPYFBSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRVGU4TSNZYGM. You are receiving this because you authored the thread.Message ID: @.***>

tez011 avatar Feb 27 '24 01:02 tez011

I can create a new branch with the exact same changes on top of latest nightly

I don't think it will change anything, you only have conflicts in the CI and docs.

ReenigneArcher avatar Feb 27 '24 01:02 ReenigneArcher

Try change this

https://github.com/LizardByte/Sunshine/blob/11c5b64d397cb51d6de755320e7134f7b01ec304/src/platform/windows/input.cpp#L1414

to

report.wButtons = static_cast<uint16_t>(ds4_buttons(gamepad_state)) | static_cast<uint16_t>(ds4_dpad(gamepad_state));

ReenigneArcher avatar Feb 27 '24 01:02 ReenigneArcher

I finally tested this, and notice a fairly substantial performance regression on your updated branch vs the first commit (b843c12ec1a533849c4fa3db01658a6d44cee229) tested on its own; my guess is the SRWLOCK changes are related.

Testing 60fps client and server with the server rendering content at 60fps, the client can only runs at ~40fps maximum. My system is not too ancient (Ryzen 5700X and RX 6600), and I'm only testing at 768p, but this system usually has no trouble keeping 60fps at 4K with the standard capture path.

Added some quick debug:

diff --git a/src/platform/windows/display_wgc.cpp b/src/platform/windows/display_wgc.cpp
index c1390136..1ccd9cb0 100644
--- a/src/platform/windows/display_wgc.cpp
+++ b/src/platform/windows/display_wgc.cpp
@@ -147,9 +147,10 @@ namespace platf::dxgi {
     AcquireSRWLockExclusive(&frame_lock);
     if (SleepConditionVariableSRW(&frame_present_cv, &frame_lock, timeout.count(), 0) == FALSE) {
       ReleaseSRWLockExclusive(&frame_lock);
-      if (GetLastError() == ERROR_TIMEOUT)
+      if (GetLastError() == ERROR_TIMEOUT) {^M
+        BOOST_LOG(error) << "timeout: " << timeout.count();^M
         return capture_e::timeout;
-      else
+      } else^M
         return capture_e::error;
     }
     if (produced_frame) {

This shows a LOT of zero count timeouts on the latest branch (and the host is definitely rendering content at 60fps, so the volume of timeouts I'm seeing should not be happening).

I can raise the capture to 60fps via one of the following steps (performed separately):

  • Using only the original commit (and capture = uwp).
  • Hardcoding the default timeout from 0ms to 16ms to correspond to a 60fps frame interval.
  • Increasing the client framerate to >60fps (such as 120fps), even though my host and client client screens only supports 60fps.

psyke83 avatar Feb 29 '24 21:02 psyke83

@psyke83 Thank you for reporting this. I noticed it too, but I thought it might be a personal problem (i.e. "It only doesn't work on my machine")

The issue no longer appears for me after applying the commit "When the frame timeout is zero, use the most recently produced frame instead of waiting for a new one." If you get a chance, I'd be curious to hear your thoughts on the actual change and if it works for you too!

tez011 avatar Mar 01 '24 01:03 tez011

@tez011

Capture is now running at 60fps on my system - nice work!

Unfortunately, whilst the incoming framerate stays at ~60, there is some noticeable periodic stuttering in games running at 60fps that isn't present in the regular capture path... but I may have discovered the solution already. My GPU/driver combo doesn't support HAGS, meaning that Sunshine is using realtime gpu scheduling on my system. This normally works well (especially to prevent stuttering when the GPU is heavily loaded), but it seems to conversly cause stuttering with your new WGC capture path. Manually setting the priority to high seems to result in capture as smooth as the DDAPI path (but I need to test more thoroughly to say that the stuttering is 100% alleviated).

I would recommend you and any other testers to check realtime vs high gpu scheduling with this capture path, just in case the issue is not unique to my specific setup. Thanks, and again - great work!

psyke83 avatar Mar 01 '24 02:03 psyke83

Rebased after successful CI run.

tez011 avatar Mar 04 '24 18:03 tez011

@cgutman

Fixed frame rate capture

This is a negative for those depending on Sunshine and Moonlight to support VRR, which apparently is working in Wayland and Gamescope at the moment. If we move to a fixed frame rate capture, then VRR would break for those users... is there any way we can make the fixed frame rate capture optional?

Nonary avatar Mar 05 '24 14:03 Nonary

I am not able to get it to work on my machine.

Error: Screen capture is not supported on this device for this release of Windows: -2147023836

Here is what I have, should it not be working or do I need a server edition of windows?

Edition Windows 11 Pro Version 23H2 Installed on ‎11/‎7/‎2023 OS build 22631.3155 Experience Windows Feature Experience Pack 1000.22684.1000.0

Nonary avatar Mar 05 '24 15:03 Nonary

Hi @Nonary, thanks for posting. I realize the error message you got is pretty vague and doesn't tell us exactly which API call doesn't work. I've pushed a change to improve that logging a bit. I also hope that when this happened Sunshine was able to fall back to the default DDAPI!

The error code you received is more properly 0x80070424 = ERROR_SERVICE_DOES_NOT_EXIST. This is puzzling, since my machine also has

Edition Windows 11 Pro Version 23H2 Installed on 2/‎8/‎2024 OS build 22631.3155 Experience Windows Feature Experience Pack 1000.22684.1000.0

tez011 avatar Mar 05 '24 17:03 tez011

Hi @cgutman,

I think #2171 breaks this branch (black screen), so I've rebased it once again. I don't have an HDR display, so any change I make risks re-introducing #2163. I do have some thoughts, but I feel this isn't the place for them and that PR is already merged anyway.

Other than this, is there anything else you think I should address before the pull request can be integrated?

tez011 avatar Mar 05 '24 17:03 tez011

Hi @Nonary, thanks for posting. I realize the error message you got is pretty vague and doesn't tell us exactly which API call doesn't work. I've pushed a change to improve that logging a bit. I also hope that when this happened Sunshine was able to fall back to the default DDAPI!

The error code you received is more properly 0x80070424 = ERROR_SERVICE_DOES_NOT_EXIST. This is puzzling, since my machine also has

Edition Windows 11 Pro Version 23H2 Installed on 2/‎8/‎2024 OS build 22631.3155 Experience Windows Feature Experience Pack 1000.22684.1000.0

You are correct, that is the error code I get, and yes it does properly fail back I just don't know why it doesn't work in my case.

Nonary avatar Mar 05 '24 20:03 Nonary

That's good news. Would you mind trying the latest version after the commit I pushed this morning? It may provide a little more information as to what specifically your OS claims is unsupported. I hate to be that guy, but I usually neglect it so it might be worth asking: what does Windows Update say?

tez011 avatar Mar 05 '24 20:03 tez011

That's good news. Would you mind trying the latest version after the commit I pushed this morning? It may provide a little more information as to what specifically your OS claims is unsupported. I hate to be that guy, but I usually neglect it so it might be worth asking: what does Windows Update say?

It doesnt really tell me anything:

[2024:03:05:15:53:19]: Info: Sunshine version: 0.21.0.f7d2769.dirty
[2024:03:05:15:53:19]: Info: nvprefs: Opened undo file from previous improper termination
[2024:03:05:15:53:19]: Info: nvprefs: Restored OGL_CPL_PREFER_DXPRESENT for base profile
[2024:03:05:15:53:19]: Info: nvprefs: Restored global profile settings from undo file - deleting the file
[2024:03:05:15:53:19]: Info: nvprefs: No need to modify application profile settings
[2024:03:05:15:53:19]: Info: nvprefs: Changed OGL_CPL_PREFER_DXPRESENT to OGL_CPL_PREFER_DXPRESENT_PREFER_ENABLED for base profile
[2024:03:05:15:53:19]: Info: Compiling shaders...
[2024:03:05:15:53:19]: Info: System tray created
[2024:03:05:15:53:19]: Info: Compiled shaders
[2024:03:05:15:53:19]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2024:03:05:15:53:19]: Info: Trying encoder [nvenc]
[2024:03:05:15:53:19]: Info: ddprobe.exe [1] [] returned: 0x887A0004
[2024:03:05:15:53:20]: Info: ddprobe.exe [2] [] returned: 0x00000000
[2024:03:05:15:53:20]: Info: Set GPU preference: 2
[2024:03:05:15:53:20]: Info: 
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00002684
Device Video Mem   : 24142 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 32651 MiB
Feature Level      : 0x0000B100
Capture size       : 3840x2160
Offset             : 0x0
Virtual Desktop    : 3840x2160
[2024:03:05:15:53:20]: Info: Active GPU has HAGS enabled
[2024:03:05:15:53:20]: Info: Using realtime GPU priority
[2024:03:05:15:53:20]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 12
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.150391,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.01 nits
Max Luminance      : 1499 nits
Max Full Luminance : 799 nits
[2024:03:05:15:53:20]: Error: Screen capture is not supported on this device for this release of Windows: failed to acquire device: [0x80070424]
[2024:03:05:15:53:20]: Info: 
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00002684
Device Video Mem   : 24142 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 32651 MiB
Feature Level      : 0x0000B100
Capture size       : 3840x2160
Offset             : 0x0
Virtual Desktop    : 3840x2160
[2024:03:05:15:53:20]: Info: Active GPU has HAGS enabled
[2024:03:05:15:53:20]: Info: Using realtime GPU priority
[2024:03:05:15:53:20]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 12
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.150391,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.01 nits
Max Luminance      : 1499 nits
Max Full Luminance : 799 nits
[2024:03:05:15:53:20]: Info: Desktop resolution [3840x2160]
[2024:03:05:15:53:20]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:03:05:15:53:20]: Info: Display refresh rate [119.88Hz]
[2024:03:05:15:53:20]: Info: Requested frame rate [60fps]
[2024:03:05:15:53:20]: Info: SDR color coding [Rec. 601]
[2024:03:05:15:53:20]: Info: Color depth: 8-bit
[2024:03:05:15:53:20]: Info: Color range: [JPEG]
[2024:03:05:15:53:20]: Info: NvEnc: created encoder P1 two-pass rfi
[2024:03:05:15:53:20]: Info: SDR color coding [Rec. 601]
[2024:03:05:15:53:20]: Info: Color depth: 8-bit
[2024:03:05:15:53:20]: Info: Color range: [JPEG]
[2024:03:05:15:53:20]: Info: NvEnc: created encoder P1 two-pass rfi
[2024:03:05:15:53:20]: Info: SDR color coding [Rec. 601]
[2024:03:05:15:53:20]: Info: Color depth: 8-bit
[2024:03:05:15:53:20]: Info: Color range: [JPEG]
[2024:03:05:15:53:20]: Info: NvEnc: created encoder P1 two-pass rfi
[2024:03:05:15:53:20]: Info: 
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00002684
Device Video Mem   : 24142 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 32651 MiB
Feature Level      : 0x0000B100
Capture size       : 3840x2160
Offset             : 0x0
Virtual Desktop    : 3840x2160
[2024:03:05:15:53:20]: Info: Active GPU has HAGS enabled
[2024:03:05:15:53:20]: Info: Using realtime GPU priority
[2024:03:05:15:53:20]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 12
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.150391,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.01 nits
Max Luminance      : 1499 nits
Max Full Luminance : 799 nits
[2024:03:05:15:53:20]: Error: Screen capture is not supported on this device for this release of Windows: failed to acquire device: [0x80070424]
[2024:03:05:15:53:20]: Info: 
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00002684
Device Video Mem   : 24142 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 32651 MiB
Feature Level      : 0x0000B100
Capture size       : 3840x2160
Offset             : 0x0
Virtual Desktop    : 3840x2160
[2024:03:05:15:53:20]: Info: Active GPU has HAGS enabled
[2024:03:05:15:53:20]: Info: Using realtime GPU priority
[2024:03:05:15:53:20]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 12
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.150391,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.01 nits
Max Luminance      : 1499 nits
Max Full Luminance : 799 nits
[2024:03:05:15:53:20]: Info: Desktop resolution [3840x2160]
[2024:03:05:15:53:20]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:03:05:15:53:20]: Info: Display refresh rate [119.88Hz]
[2024:03:05:15:53:20]: Info: Requested frame rate [60fps]
[2024:03:05:15:53:20]: Info: SDR color coding [Rec. 709]
[2024:03:05:15:53:20]: Info: Color depth: 10-bit
[2024:03:05:15:53:20]: Info: Color range: [JPEG]
[2024:03:05:15:53:20]: Info: NvEnc: created encoder P1 10-bit two-pass rfi
[2024:03:05:15:53:20]: Info: SDR color coding [Rec. 709]
[2024:03:05:15:53:20]: Info: Color depth: 10-bit
[2024:03:05:15:53:20]: Info: Color range: [JPEG]
[2024:03:05:15:53:20]: Info: NvEnc: created encoder P1 10-bit two-pass rfi
[2024:03:05:15:53:20]: Info: 
[2024:03:05:15:53:20]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:03:05:15:53:20]: Info: 
[2024:03:05:15:53:20]: Info: Found H.264 encoder: h264_nvenc [nvenc]
[2024:03:05:15:53:20]: Info: Found HEVC encoder: hevc_nvenc [nvenc]
[2024:03:05:15:53:20]: Info: Found AV1 encoder: av1_nvenc [nvenc]
[2024:03:05:15:53:21]: Info: Registered Sunshine mDNS service
[2024:03:05:15:53:21]: Info: Opening UI from system tray
[2024:03:05:15:53:21]: Info: Resolved user-provided command 'https://localhost:47990' to '"C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe" --single-argument https://localhost:47990'
[2024:03:05:15:53:21]: Info: https://localhost:47990 running with PID 8824
[2024:03:05:15:53:21]: Info: Opened url [https://localhost:47990]
[2024:03:05:15:53:44]: Info: Encoder reenumeration is required
[2024:03:05:15:53:44]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2024:03:05:15:53:44]: Info: Trying encoder [nvenc]
[2024:03:05:15:53:44]: Info: 
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00002684
Device Video Mem   : 24142 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 32651 MiB
Feature Level      : 0x0000B100
Capture size       : 3840x2160
Offset             : 0x0
Virtual Desktop    : 3840x2160
[2024:03:05:15:53:44]: Info: Active GPU has HAGS enabled
[2024:03:05:15:53:44]: Info: Using realtime GPU priority
[2024:03:05:15:53:44]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 12
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.150391,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.01 nits
Max Luminance      : 1499 nits
Max Full Luminance : 799 nits
[2024:03:05:15:53:44]: Error: Screen capture is not supported on this device for this release of Windows: failed to acquire device: [0x80070424]
[2024:03:05:15:53:44]: Info: 
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00002684
Device Video Mem   : 24142 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 32651 MiB
Feature Level      : 0x0000B100
Capture size       : 3840x2160
Offset             : 0x0
Virtual Desktop    : 3840x2160
[2024:03:05:15:53:44]: Info: Active GPU has HAGS enabled
[2024:03:05:15:53:44]: Info: Using realtime GPU priority
[2024:03:05:15:53:44]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 12
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.150391,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.01 nits
Max Luminance      : 1499 nits
Max Full Luminance : 799 nits
[2024:03:05:15:53:44]: Info: Desktop resolution [3840x2160]
[2024:03:05:15:53:44]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:03:05:15:53:44]: Info: Display refresh rate [119.88Hz]
[2024:03:05:15:53:44]: Info: Requested frame rate [60fps]
[2024:03:05:15:53:44]: Info: SDR color coding [Rec. 601]
[2024:03:05:15:53:44]: Info: Color depth: 8-bit
[2024:03:05:15:53:44]: Info: Color range: [JPEG]
[2024:03:05:15:53:44]: Info: NvEnc: created encoder P1 two-pass rfi
[2024:03:05:15:53:44]: Info: SDR color coding [Rec. 601]
[2024:03:05:15:53:44]: Info: Color depth: 8-bit
[2024:03:05:15:53:44]: Info: Color range: [JPEG]
[2024:03:05:15:53:44]: Info: NvEnc: created encoder P1 two-pass rfi
[2024:03:05:15:53:44]: Info: SDR color coding [Rec. 601]
[2024:03:05:15:53:44]: Info: Color depth: 8-bit
[2024:03:05:15:53:44]: Info: Color range: [JPEG]
[2024:03:05:15:53:44]: Info: NvEnc: created encoder P1 two-pass rfi
[2024:03:05:15:53:44]: Info: 
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00002684
Device Video Mem   : 24142 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 32651 MiB
Feature Level      : 0x0000B100
Capture size       : 3840x2160
Offset             : 0x0
Virtual Desktop    : 3840x2160
[2024:03:05:15:53:44]: Info: Active GPU has HAGS enabled
[2024:03:05:15:53:44]: Info: Using realtime GPU priority
[2024:03:05:15:53:44]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 12
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.150391,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.01 nits
Max Luminance      : 1499 nits
Max Full Luminance : 799 nits
[2024:03:05:15:53:44]: Error: Screen capture is not supported on this device for this release of Windows: failed to acquire device: [0x80070424]
[2024:03:05:15:53:44]: Info: 
Device Description : NVIDIA GeForce RTX 4090
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00002684
Device Video Mem   : 24142 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 32651 MiB
Feature Level      : 0x0000B100
Capture size       : 3840x2160
Offset             : 0x0
Virtual Desktop    : 3840x2160
[2024:03:05:15:53:44]: Info: Active GPU has HAGS enabled
[2024:03:05:15:53:44]: Info: Using realtime GPU priority
[2024:03:05:15:53:44]: Info: 
Colorspace         : DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
Bits Per Color     : 12
Red Primary        : [0.639648,0.330078]
Green Primary      : [0.299805,0.599609]
Blue Primary       : [0.150391,0.0595703]
White Point        : [0.3125,0.329102]
Min Luminance      : 0.01 nits
Max Luminance      : 1499 nits
Max Full Luminance : 799 nits
[2024:03:05:15:53:44]: Info: Desktop resolution [3840x2160]
[2024:03:05:15:53:44]: Info: Desktop format [DXGI_FORMAT_B8G8R8A8_UNORM]
[2024:03:05:15:53:44]: Info: Display refresh rate [119.88Hz]
[2024:03:05:15:53:44]: Info: Requested frame rate [60fps]
[2024:03:05:15:53:44]: Info: SDR color coding [Rec. 709]
[2024:03:05:15:53:44]: Info: Color depth: 10-bit
[2024:03:05:15:53:44]: Info: Color range: [JPEG]
[2024:03:05:15:53:44]: Info: NvEnc: created encoder P1 10-bit two-pass rfi
[2024:03:05:15:53:44]: Info: SDR color coding [Rec. 709]
[2024:03:05:15:53:44]: Info: Color depth: 10-bit
[2024:03:05:15:53:44]: Info: Color range: [JPEG]
[2024:03:05:15:53:45]: Info: NvEnc: created encoder P1 10-bit two-pass rfi
[2024:03:05:15:53:45]: Info: 
[2024:03:05:15:53:45]: Info: // Ignore any errors mentioned above, they are not relevant. //
[2024:03:05:15:53:45]: Info: 
[2024:03:05:15:53:45]: Info: Found H.264 encoder: h264_nvenc [nvenc]
[2024:03:05:15:53:45]: Info: Found HEVC encoder: hevc_nvenc [nvenc]
[2024:03:05:15:53:45]: Info: Found AV1 encoder: av1_nvenc [nvenc]
[2024:03:05:15:53:45]: Info: Executing Do Cmd: [powershell.exe -executionpolicy bypass -file "E:\sources\MonitorSwapAutomation\MonitorSwapper.ps1"]
[2024:03:05:15:53:45]: Info: powershell.exe -executionpolicy bypass -file "E:\sources\MonitorSwapAutomation\MonitorSwapper.ps1" running with PID 14732
[2024:03:05:15:53:49]: Info: Executing Do Cmd: [powershell.exe -executionpolicy bypass -file "E:\sources\AutoHDRSwitch\HDRToggle.ps1"]
[2024:03:05:15:53:49]: Info: powershell.exe -executionpolicy bypass -file "E:\sources\AutoHDRSwitch\HDRToggle.ps1" running with PID 27660
[2024:03:05:15:53:50]: Info: Executing Do Cmd: [powershell.exe -executionpolicy bypass -file "E:\sources\AudioSwapper\AudioSwapper.ps1"]
[2024:03:05:15:53:50]: Info: powershell.exe -executionpolicy bypass -file "E:\sources\AudioSwapper\AudioSwapper.ps1" running with PID 20880
[2024:03:05:15:53:52]: Info: Executing Do Cmd: [powershell.exe -executionpolicy bypass -WindowStyle Hidden -file "E:\sources\ResolutionAutomation\ResolutionMatcher.ps1"]
[2024:03:05:15:53:52]: Info: powershell.exe -executionpolicy bypass -WindowStyle Hidden -file "E:\sources\ResolutionAutomation\ResolutionMatcher.ps1" running with PID 27332
[2024:03:05:15:53:53]: Info: Executing [Desktop]

Windows Update Say?

You already know I am on the latest updates for everything since I have the 22631.3155 build.

Nonary avatar Mar 05 '24 21:03 Nonary

@Nonary

I believe that the error you're seeing is due to @cgutman's observation here that the code is not compatible with our service model (as well as UAC, as-is). You probably need to stop the service and run the executable manually in order to test the new capture path.

A simple batch script like the following would work (but needs elevation to control service stop):

cd /d "c:\program files\sunshine"
net stop sunshineservice
sunshine.exe

@tez011: OT, but I'm currently stuck with Wayland on my client machine due to X11 breakage, so it's difficult to test the stutter issue related to realtime scheduling; Wayland itself causes some frame drops particularly for video playback apps on my system. Don't worry about addressing that particular issue, but as I said, it might be worth checking if you or others see any stuttering issues currently.

psyke83 avatar Mar 05 '24 22:03 psyke83

This is a negative for those depending on Sunshine and Moonlight to support VRR, which apparently is working in Wayland and Gamescope at the moment. If we move to a fixed frame rate capture, then VRR would break for those users... is there any way we can make the fixed frame rate capture optional?

We'll have to see how VRR displays actually behave with this new API. DXGI DDA won't be going away anytime soon, so users will always have the option to use that instead. But I think fixed rate capture should be the default, since bitrate calculations go awry when we tell the encoder that we're encoding N FPS and then encode N/2 (you get only half the configure bitrate). Some clients (particularly on Android) also seem to hold at least one frame in the decoder, which means lower frame rates translate directly into higher decode latency on the client.

Other than this, is there anything else you think I should address before the pull request can be integrated?

Other than the breakage with https://github.com/LizardByte/Sunshine/pull/2171 (which I've noted the fix for you), I think the PR code itself is fine. We're not merging major changes yet because v0.22 just shipped and we're still gathering fixes to release in v0.22.1. Once merging is fully open again, we can merge this.

We'll need some additional changes on top to get this fully production-ready after merge like:

  • Fixing WGC to work properly when running as a service
  • Avoiding DXGI DDA functionality checks when running with WGC
  • Handling cross-adapter encoding better (enlighten the encoder selection to be aware of the cost of using a cross-adapter hardware encoder vs a local one)

cgutman avatar Mar 06 '24 03:03 cgutman

bump bump merge

Originalimoc avatar Mar 24 '24 07:03 Originalimoc

I've just updated the branch to contain nightly changes since 0.22.2. I was planning to wait until the release was clear for a couple of weeks but I'm ready to merge if you'll have it!

tez011 avatar Mar 24 '24 17:03 tez011

@tez011 thank you! This is definitely one of the ones we want to get reviewed and merged soon.

ReenigneArcher avatar Mar 24 '24 18:03 ReenigneArcher

I have problems with using MSYS UCRT64. Currently the std::regex is just broken in it.

It takes 1 min 20 sec to parse a single regex for HTTP server:

[2024:03:25:20:41:05]: Info: Registered Sunshine mDNS service
[2024:03:25:20:42:23]: Info: Configuration UI available at [https://localhost:47990]

With my PR (https://github.com/LizardByte/Sunshine/pull/2032) it would just throw an exception if std::regex was used from a thread (I just use boost::regex, so that's not a problem).

We could define USE_BOOST_REGEX for the HTTP server and that would eliminate all relevant usages (for now), but it is still a problem for the future.

Would it be smarter to migrate to Microsoft build tools? What do you think @ReenigneArcher?

FrogTheFrog avatar Mar 25 '24 18:03 FrogTheFrog

@FrogTheFrog I'm not sold on migrating to different tooling. Mostly because I've never used anything on Windows except msys2.

According to https://github.com/msys2/MINGW-packages/pull/20003#issuecomment-1936769612 UCRT64 is the same as MINGW64 ... maybe the issue is with c++20 instead?

ReenigneArcher avatar Mar 25 '24 21:03 ReenigneArcher