steam-for-linux
steam-for-linux copied to clipboard
ValveSoftware
Steam In-Home Streaming is slow with Vulkan-based games
Your system information
- Steam client version (build number or date): Jul 24 2018, at 18:27:28
- Distribution (e.g. Ubuntu): Arch
- Opted into Steam client beta?: [Yes/No] No
- Have you checked for system updates?: [Yes/No] Yes
Please describe your issue in as much detail as possible:
Whenever I try to stream Vulkan games to my Steam Link I get considerably lower FPS.
Case 1: Rise of the Tomb Raider Enabling the streaming performance overlay, I can see that it shows "Slow Capture". FPS is borderline unplayable at 20-ish FPS.
Case 2: Mad Max I tried both OpenGL and Vulkan with this game. Without in-home streaming, Vulkan vastly outperforms OpenGL. However, when I try to stream the game in Vulkan mode, Steam Link reports a 20 FPS drop and the gameplay stutters quite a bit. Streaming with OpenGL mode seems relatively normal and smoother, even though its inherently slower than Vulkan.
All my other games are OpenGL based and they run great with Steam Link.
Steps for reproducing this issue:
- Run either Rise of the Tomb Raider or Mad Max in Vulkan mode
- Stream to Steam Link
- Notice stuttering and FPS drops
The video card you have, last I checked, is not used by steam for hardware encoding because steam does not use the linux standard hardware encoding interfaces (instead it uses some horrible nvidia-specific ones for who knows what reasons), consequently it is having to pull back the data to the CPU for processing every single frame. I'd imagine this would be fixed once Steam finally uses the standard interfaces for hardware encoding (which would work with amd/intel/nvidia all).
@OvermindDL1 Yes, I am aware that Steam currently only supports NVENC and not VA-API for hardware encoding. I am currently using libx264 software encoding to do the streaming. However, I do not think that's the issue here, since OpenGL games stream just fine with the software encoding. This bug is specific to Vulkan games. The capturing of the game is what's slow here, not the encoding.
I'd wager it's how the capturing has to pull the data over, not looked in to it yet though...
Still an issue with the Nov. 10th steam client.
This is not a hardware / software resource limitation. It's some incompatibility with steam capturing Vulkan output at high frame rates. I, too, see this with a Vulkan game and NOT with OpenGL games. It feels like the steam client doesn't know if Vulkan is outputting high FPS or not? I wish I could see the source code to figure this one out.
Example: Mad Max - The host PC is playing the game 60 fps + while the client PC reports it is receiving 59.97 fps but the actual screen looks like 30 fps or lower. If I open up the game map (fullscreen) the actual FPS jumps up to 60 fps and moving the game map is smooth like I would expect. Exiting the game map drops down the drawn framerate to 30fps or lower and the host PC is still humming along at 60 fps+
if it makes you feel any better, im on windows and cant stream vulkan at all. anything that uses vulkan = black screen or freeze.
this is with a 2080ti. has been like that for ages. I dont even try anymore. Some people have it working, no idea whats different but seems exceptionally finicky relative to what it provides.
still i'd like the option.
just to add to this. vulkan will stream if you change your buffer flipping mode from the standard/default page flip to block transfer via the OGL force BLIT ON flag in nvidia profile inspector.
of course this creates all kinds of other issues.
I contacted and presented support with this and the respnose was curt "we do not support the vulkan api at this time"
Any news on this issue? streaming Vulkan games still tanks my performance even if the game is basic 2D
I'm also having this issue still, on an Nvidia GTX 1080 with the 510.47.03 drivers. I'm running Arch with the 5.16.7 kernel with the "Zen" patchset (though I've observed it with a vanilla Arch kernel). What I'm noticing is the Steam client itself has very high CPU usage when the issue is occurring, using most (around 70-80%) of one thread's time. It's also using Game Vulkan RGB + libyuv + NVENC H264. I also find it odd that the latency graph shows occasional bursts of less and more consistent latency. (I'm on Wi-Fi on my client, so I expect some inconsistency always) Example screenshot (though this was during a scene transition so the overlay reports low game framerate as well): 
As soon as I pull away from the game (specifically I'm launching glxgears over SSH), and the stream switches to desktop capture, using Desktop OpenGL NV12 + NVENC, performance is much better, and the Steam client CPU's usage drops, and is more evenly spread out across CPU threads.
So is there some potential slowness in the libyuv conversion or ingesting video from the overlay as a Vk layer?
I have a very similar issue. If steam remote play defaults to the vulkan rgb driver then the graph jumps to 100ms. If it defaults to OpenGL NV12 it runs like a dream. You can manipulate it to run using OpenGL by running the game by detaching the window from fullscreen (inconsistent), you can also force it to use OpenGl as the driver with PROTON_USE_WINED3D11=1 which obviously then crashes a lot of games.
Any input welcome,
Thanks
I am experiencing this issue with Detroit Become Human and my Steam Link.
Game launches and i can hear audio but the game isn't visible. Using a keyboard I can alt tab to the game but it's effectively frozen.
I have to run the game in windowed mode in order to play it and performance isn't great. I have to run at medium settings with a gtx1070
Yeah, this is totally broken.
OBS works without issue so this has to be due to the way steam is doing the capture
for some reason Halo MCC is captured with OpenGL even though it is running through proton + dxvk, and i get decent capture performance. if i run Halo MCC through gamescope, it is captured with Vulkan and performance is terrible, like with most other games
played around a bit more, and it seems when streaming Halo MCC without gamescope in windowed mode, steam will always capture the full desktop, and not just the game window. i guess this is also happening when Halo is in fullscreen, and is some defect that in this situation is beneficial for streaming performance. now if we could find a way to trigger this defect intentionally...
Arch 5.16 PDS TKG, nvidia 510.54
Noticed this in elden ring most recently, but have noticed something in a game called cloudpunk.
If the game is in fullscreen mode, the fps drops severely to 24fps.
If I toggle to windowed mode, it's around 50 fps - still not perfect, but odd that it doubles in windowed mode when fullscreen exclusive should technically perform better.
In Windowed the encoder is "game vulkan rgb + libyuv + NVENC H264"
In fullscreen I also get "scale" added before NVENC. Probably because my desktop resolution is higher than the streaming client device (2560x1440 on desktop, 1920x1080 on client).
I've noticed that although I have "change resolution to match client" enabled in remote play settings on host, the resolution doesn't get changed.
So I'm wondering if this is something to do with the proton fullscreen hack that scales output instead of changing resolution.
Edit: and yup, forcing opengl with PROTON_USE_WINED3D=1 gets 60 fps in fullscreen and windowed using opengl nv12 encoder. So it's something to do with at least vulkan capture mode, if not vkd3d itself
@lewiji The issue is the libyuv load on the cpu. And libscale too. Both use CPU for image manipulation and max out very quicjky
@lewiji The issue is the libyuv load on the cpu. And libscale too. Both use CPU for image manipulation and max out very quicjky
I see - and currently there's no way around this?
I noticed that even forcing opengl mode, even though the overlay was consistently reporting 60fps, the game felt like it was running much lower at times, so I'm guessing it didn't really fix the issue.
I can confirm this issue. In my case, it was introduced with the latest update of the Steam client on 4th or 5th of March. Before the update, I had consistent capture rates of 60 fps (at 1920x1080) in all games. Now, all Proton games (using Vulkan based encoder) have capture rates between 30 and 40 fps. Native games (e.g. Alien:Isolation, Dungeons III) still work fine with opengl encoder.
CPU: AMD Ryzen 7 3700X 8-Core AMD Radeon RX 580 Series (POLARIS10, DRM 3.41.0, 5.13.0-30-generic, LLVM 12.0.1) Kubuntu 21.10 (64 Bit) Kernel version: 5.13.0-30-generic
@lewiji I wish. I've been searching for one..
I've found that forcing OpenGL causes things to be fine, but I guess this is a YMMV situation. Not that I can for it to stay in OpenGL mode. Which is silly because in a fully composited desktop everything should be available as OpenGL.
I'm more annoyed that NVFBC doesn't work at all on linux anymore. Patched driver or no.
I have a similar issue, I tried dragons dogma, which works wonderfully in steam play but gives low fps with remote play. Celeste's native version that I got from itch worked perfectly fine. I have a Ryzen + RTX2070 Setup on Arch with nvidia drivers.
This is happening to me too. Rx6700xt, arch Linux, using vaapi.
I noticed that as soon as a vulkan game is used, the encoder changes to vulkan rgb + yuv420 + vaapi.
I also noticed in the console logs that something is setting the target framerate to 30 and locking it.
It's something specific to vulkan rgb + yuv420. I can alt tab out of a game, have it running in the background, and the encoder switches back to desktop gl, and 60fps, so it's not a resource issue.
Here are some console logs, where you can see the framerate set to 30. Any game at all that switches to vulkan rgb, yuv420, in the encoder info, no matter what, is locked to thirty fps or so. A game that doesn't use this encoder mode, and is graphically far more demanding, does not have this issue.
ffmpeg verbose: Input surface format is nv12. ffmpeg verbose: Using VAAPI profile VAProfileH264Main (6).
ffmpeg verbose: Using VAAPI entrypoint VAEntrypointEncSlice (6). ffmpeg verbose: Using VAAPI render target format YUV420 (0x1).
ffmpeg verbose: RC mode: VBR. ffmpeg verbose: RC target: 83% of 12000000 bps over 1000 ms. ffmpeg verbose: RC buffer: 12000000 bits, initial fullness 9000000 bits.
ffmpeg verbose: RC framerate: 239/4 (59.75 fps). ffmpeg verbose: Using intra and P-frames (supported references: 1 / 0). ffmpeg warning: Driver does not support some wanted packed headers (wanted 0xd, found 0x1). ffmpeg verbose: Using level 4. >>> Capture method set to Game Vulkan RGB + libyuv + VAAPI H264 COpenGLSurface::BInitialize: Current session is 6 Installing breakpad exception handler for appid(steam)/version(1647446817) Installing breakpad exception handler for appid(steam)/version(1647446817) Installing breakpad exception handler for appid(steam)/version(1647446817) Installing breakpad exception handler for appid(steam)/version(1647446817)
Installing breakpad exception handler for appid(steam)/version(1647446817) CLIENT: Got control packet k_EStreamControlVideoEncoderInfo
DispatchAsyncEvent backlog, failed to dispatch all this frame. Queue depth: 905 (907 input number was) (steam:74944): Gtk-WARNING **: 18:33:36.900: gtk_disable_setlocale() must be called before gtk_init() Slow image load - /home/hwkiller/.local/share/@Steam/tenfoot/resource/images/profile/activity_bg_wash.png (dimensions 1879x928, took 30 msec) Got control packet k_EStreamControlGetTouchConfigData Getting touch control config for Adding action set Default Adding input switch switches - button_escape for gamepad action 11 Adding input switch switches - button_menu for gamepad action 12
Adding input switch switches - left_bumper for gamepad action 7 Adding input switch switches - right_bumper for gamepad action 8 Adding input switch switches - button_back_left for gamepad action 3
Adding input switch switches - button_back_right for gamepad action 6 Adding source button_diamond four_buttons Adding input button_diamond four_buttons - button_a for gamepad action 3
Adding input button_diamond four_buttons - button_b for gamepad action 4 Adding input button_diamond four_buttons - button_x for gamepad action 6
Adding input button_diamond four_buttons - button_y for gamepad action 5
Adding source joystick joystick_move Adding input joystick joystick_move - click for gamepad action 9
Adding source left_trigger trigger Adding input left_trigger trigger - click for gamepad action 1 Adding source right_trigger trigger
Adding input right_trigger trigger - click for gamepad action 2
Adding source right_joystick joystick_move Adding input right_joystick joystick_move - click for gamepad action 10
Adding source dpad dpad Adding input dpad dpad - dpad_north for gamepad action 14 Adding input dpad dpad - dpad_south for gamepad action 15
Adding input dpad dpad - dpad_east for gamepad action 17 Adding input dpad dpad - dpad_west for gamepad action 16
Got control packet k_EStreamControlGetTouchConfigData Getting touch control config for Adding action set Default
Adding input switch switches - button_escape for gamepad action 11 Adding input switch switches - button_menu for gamepad action 12 Adding input switch switches - left_bumper for gamepad action 7
Adding input switch switches - right_bumper for gamepad action 8 Adding input switch switches - button_back_left for gamepad action 3 Adding input switch switches - button_back_right for gamepad action 6 Adding source button_diamond four_buttons Adding input button_diamond four_buttons - button_a for gamepad action 3 Adding input button_diamond four_buttons - button_b for gamepad action 4
Adding input button_diamond four_buttons - button_x for gamepad action 6 Adding input button_diamond four_buttons - button_y for gamepad action 5
Adding source joystick joystick_move Adding input joystick joystick_move - click for gamepad action 9 Adding source left_trigger trigger
Adding input left_trigger trigger - click for gamepad action 1 Adding source right_trigger trigger Adding input right_trigger trigger - click for gamepad action 2 Adding source right_joystick joystick_move Adding input right_joystick joystick_move - click for gamepad action 10 Adding source dpad dpad Adding input dpad dpad - dpad_north for gamepad action 14 Adding input dpad dpad - dpad_south for gamepad action 15 Adding input dpad dpad - dpad_east for gamepad action 17 Adding input dpad dpad - dpad_west for gamepad action 16
CLIENT: Got control packet k_EStreamControlTouchConfigActive
CLIENT: Got control packet k_EStreamControlTouchConfigActive Slow image load - /home/hwkiller/.local/share/@Steam/tenfoot/resource/images/mainmenu_bg_2.png (dimensions 1920x1080, took 67 msec)
CLIENT: Video rect: 1920x1080 at 180,0 CLIENT: Texture rect: 1920x1080 at 0,0 CLIENT: Got control packet k_EStreamControlSetTouchConfigData CLIENT: Ignoring duplicate controller configuration for 3771021810, revision 0 CLIENT: Got control packet k_EStreamControlSetTouchConfigData Slow image load - /home/hwkiller/.local/share/@Steam/tenfoot/resource/images/profile/profile_bg_wash.png (dimensions 1879x928, took 45 msec) Slow image load - /home/hwkiller/.local/share/@Steam/tenfoot/resource/images/panel_background.png (dimensions 1920x1080, took 44 msec) Slow image load - /home/hwkiller/.local/share/@Steam/tenfoot/resource/images/overlay/overlay_bg_wash.png (dimensions 1879x928, took 46 msec) Slow image load - /home/hwkiller/.local/share/@Steam/userdata/9936320/760/remote/10663366539924733952/screenshots/20220205143826_1.jpg (dimensions 177x100, took 47 msec) Slow image load - /home/hwkiller/.local/share/@Steam/userdata/9936320/760/remote/10663366539924733952/screenshots/20220205140847_1.jpg (dimensions 177x100, took 55 msec)
Slow image load - /home/hwkiller/.local/share/@Steam/tenfoot/resource/images/background_alpha_mask.png (dimensions 1920x1080, took 24 msec) Slow image load - /home/hwkiller/.local/share/@Steam/userdata/9936320/760/remote/10663366539924733952/screenshots/20220113230756_1.jpg (dimensions 177x100, took 50 msec) HTTP Response for https://steamcommunity-a.akamaihd.net/news/newsforapp/v0001/?appid=3771021810&count=5&maxlength=500&format=json: 403, 0 HTTP Response failed https://steamcommunity-a.akamaihd.net/news/newsforapp/v0001/?appid=3771021810&count=5&maxlength=500&format=json Bitrate adapter: state = 2, minRTT = 3, maxRTT = 7, gain = 1.25, bitrate = 12500
Setting target framerate: 59.75 [timing: game 6.32, capture 1.51, convert 22.65, encode 10.60, network 2.27, decode 2.19, display 14.02] CLIENT: Targeting 29.88 FPS CLIENT: Got control packet k_EStreamControlSetTargetFramerate ffmpeg verbose: libva: VA-API version 1.13.0 ffmpeg verbose: libva: Trying to open /usr/lib32/dri/radeonsi_drv_video.so ffmpeg verbose: libva: Found init function __vaDriverInit_1_13 ffmpeg verbose: libva: va_openDriver() returns 0 ffmpeg verbose: Initialised VAAPI connection: version 1.13 ffmpeg verbose: VAAPI driver: Mesa Gallium driver 21.3.7 for AMD Radeon RX 6700 XT (NAVY_FLOUNDER, DRM 3.44.0, 5.16.15-zen1-1-zen, LLVM 13.0.1). ffmpeg verbose: Driver not found in known nonstandard list, using standard behaviour. ffmpeg verbose: Input surface format is nv12. ffmpeg verbose: Using VAAPI profile VAProfileH264Main (6). ffmpeg verbose: Using VAAPI entrypoint VAEntrypointEncSlice (6). ffmpeg verbose: Using VAAPI render target format YUV420 (0x1). ffmpeg verbose: RC mode: VBR. ffmpeg verbose: RC target: 83% of 15000000 bps over 1000 ms.
ffmpeg verbose: RC buffer: 15000000 bits, initial fullness 11250000 bits. ffmpeg verbose: RC framerate: 239/8 (29.88 fps). ffmpeg verbose: Using intra and P-frames (supported references: 1 / 0). ffmpeg warning: Driver does not support some wanted packed headers (wanted 0xd, found 0x1). ffmpeg verbose: Using level 4. Setting target framerate: 14.94 [timing: game 4.59, capture 1.58, convert 22.60, encode 9.72, network 3.68, decode 2.24, display 9.24] CLIENT: Targeting 29.88 FPS CLIENT: Got control packet k_EStreamControlSetTargetFramerate ffmpeg verbose: libva: VA-API version 1.13.0 ffmpeg verbose: libva: Trying to open /usr/lib32/dri/radeonsi_drv_video.so ffmpeg verbose: libva: Found init function __vaDriverInit_1_13 ffmpeg verbose: libva: va_openDriver() returns 0
ffmpeg verbose: Initialised VAAPI connection: version 1.13 ffmpeg verbose: VAAPI driver: Mesa Gallium driver 21.3.7 for AMD Radeon RX 6700 XT (NAVY_FLOUNDER, DRM 3.44.0, 5.16.15-zen1-1-zen, LLVM 13.0.1).
ffmpeg verbose: Driver not found in known nonstandard list, using standard behaviour. ffmpeg verbose: Input surface format is nv12. ffmpeg verbose: Using VAAPI profile VAProfileH264Main (6). ffmpeg verbose: Using VAAPI entrypoint VAEntrypointEncSlice (6). ffmpeg verbose: Using VAAPI render target format YUV420 (0x1). ffmpeg verbose: RC mode: VBR.
ffmpeg verbose: RC target: 83% of 15000000 bps over 1000 ms. ffmpeg verbose: RC buffer: 15000000 bits, initial fullness 11250000 bits. ffmpeg verbose: RC framerate: 239/4 (59.75 fps).
ffmpeg verbose: Using intra and P-frames (supported references: 1 / 0).
ffmpeg warning: Driver does not support some wanted packed headers (wanted 0xd, found 0x1). ffmpeg verbose: Using level 4. Bitrate adapter: state = 2, minRTT = 3, maxRTT = 7, gain = 1.25, bitrate = 15625 Setting target framerate: 59.75 [timing: game 3.21, capture 1.70, convert 22.75, encode 10.10, network 2.56, decode 1.48, display 8.39] CLIENT: Targeting 29.88 FPS CLIENT: Got control packet k_EStreamControlSetTargetFramerate ffmpeg verbose: libva: VA-API version 1.13.0
ffmpeg verbose: libva: Trying to open /usr/lib32/dri/radeonsi_drv_video.so
ffmpeg verbose: libva: Found init function __vaDriverInit_1_13 ffmpeg verbose: libva: va_openDriver() returns 0 ffmpeg verbose: Initialised VAAPI connection: version 1.13
ffmpeg verbose: VAAPI driver: Mesa Gallium driver 21.3.7 for AMD Radeon RX 6700 XT (NAVY_FLOUNDER, DRM 3.44.0, 5.16.15-zen1-1-zen, LLVM 13.0.1). ffmpeg verbose: Driver not found in known nonstandard list, using standard behaviour. ffmpeg verbose: Input surface format is nv12. ffmpeg verbose: Using VAAPI profile VAProfileH264Main (6). ffmpeg verbose: Using VAAPI entrypoint VAEntrypointEncSlice (6).
ffmpeg verbose: Using VAAPI render target format YUV420 (0x1). ffmpeg verbose: RC mode: VBR. ffmpeg verbose: RC target: 83% of 18750000 bps over 1000 ms.
ffmpeg verbose: RC buffer: 18750000 bits, initial fullness 14062500 bits. ffmpeg verbose: RC framerate: 239/8 (29.88 fps). ffmpeg verbose: Using intra and P-frames (supported references: 1 / 0). ffmpeg warning: Driver does not support some wanted packed headers (wanted 0xd, found 0x1). ffmpeg verbose: Using level 4.
Bitrate adapter: state = 2, minRTT = 4, maxRTT = 6, gain = 1.25, bitrate = 19531 Setting target framerate: 14.94 [timing: game 3.43, capture 1.66, convert 22.90, encode 10.39, network 3.06, decode 1.30, display 6.64] CLIENT: Targeting 29.88 FPS
CLIENT: Got control packet k_EStreamControlSetTargetFramerate
Wow, that log got butchered from my tablet.
Let's try another one:
CLIENT: Got control packet k_EStreamControlSetActivity
>>> Switching video stream from Desktop_MovieStream to GameOverlay_MovieStream_3951
>>> Capture method set to Game Vulkan RGB + libyuv + VAAPI H264
CLIENT: Got control packet k_EStreamControlVideoEncoderInfo
CLIENT: Video rect: 1920x1080 at 180,0
CLIENT: Texture rect: 1920x1080 at 0,0
CLIENT: Got control packet k_EStreamControlTouchConfigActive
CLIENT: Got control packet k_EStreamControlTouchConfigActive
Setting target framerate: 59.75 [timing: game 3.59, capture 1.66, convert 23.59, encode 9.72, network 14.75, decode 3.23, display 7.61]
CLIENT: Targeting 29.88 FPS
CLIENT: Got control packet k_EStreamControlSetTargetFramerate
ffmpeg verbose: libva: VA-API version 1.13.0
ffmpeg verbose: libva: Trying to open /usr/lib32/dri/radeonsi_drv_video.so
ffmpeg verbose: libva: Found init function __vaDriverInit_1_13
ffmpeg verbose: libva: va_openDriver() returns 0
ffmpeg verbose: Initialised VAAPI connection: version 1.13
ffmpeg verbose: VAAPI driver: Mesa Gallium driver 21.3.7 for AMD Radeon RX 6700 XT (NAVY_FLOUNDER, DRM 3.44.0, 5.16.15-zen1-1-zen, LLVM 13.0.1).
ffmpeg verbose: Driver not found in known nonstandard list, using standard behaviour.
ffmpeg verbose: Input surface format is nv12.
ffmpeg verbose: Using VAAPI profile VAProfileH264Main (6).
ffmpeg verbose: Using VAAPI entrypoint VAEntrypointEncSlice (6).
ffmpeg verbose: Using VAAPI render target format YUV420 (0x1).
ffmpeg verbose: RC mode: VBR.
ffmpeg verbose: RC target: 83% of 36000000 bps over 1000 ms.
ffmpeg verbose: RC buffer: 36000000 bits, initial fullness 27000000 bits.
ffmpeg verbose: RC framerate: 239/8 (29.88 fps).
ffmpeg verbose: Using intra and P-frames (supported references: 1 / 0).
ffmpeg warning: Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
ffmpeg verbose: Using level 4.1.
Setting target framerate: 14.94 [timing: game 3.03, capture 1.91, convert 24.61, encode 10.36, network 6.80, decode 3.27, display 21.09]
CLIENT: Targeting 29.88 FPS
CLIENT: Got control packet k_EStreamControlSetTargetFramerate
ffmpeg verbose: libva: VA-API version 1.13.0
ffmpeg verbose: libva: Trying to open /usr/lib32/dri/radeonsi_drv_video.so
ffmpeg verbose: libva: Found init function __vaDriverInit_1_13
ffmpeg verbose: libva: va_openDriver() returns 0
ffmpeg verbose: Initialised VAAPI connection: version 1.13
ffmpeg verbose: VAAPI driver: Mesa Gallium driver 21.3.7 for AMD Radeon RX 6700 XT (NAVY_FLOUNDER, DRM 3.44.0, 5.16.15-zen1-1-zen, LLVM 13.0.1).
ffmpeg verbose: Driver not found in known nonstandard list, using standard behaviour.
ffmpeg verbose: Input surface format is nv12.
ffmpeg verbose: Using VAAPI profile VAProfileH264Main (6).
ffmpeg verbose: Using VAAPI entrypoint VAEntrypointEncSlice (6).
ffmpeg verbose: Using VAAPI render target format YUV420 (0x1).
ffmpeg verbose: RC mode: VBR.
ffmpeg verbose: RC target: 83% of 36000000 bps over 1000 ms.
ffmpeg verbose: RC buffer: 36000000 bits, initial fullness 27000000 bits.
ffmpeg verbose: RC framerate: 239/4 (59.75 fps).
ffmpeg verbose: Using intra and P-frames (supported references: 1 / 0).
ffmpeg warning: Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
ffmpeg verbose: Using level 4.1.
Setting target framerate: 59.75 [timing: game 2.51, capture 1.81, convert 23.18, encode 10.50, network 11.23, decode 3.26, display 8.21]
CLIENT: Targeting 29.88 FPS
CLIENT: Got control packet k_EStreamControlSetTargetFramerate
ffmpeg verbose: libva: VA-API version 1.13.0
ffmpeg verbose: libva: Trying to open /usr/lib32/dri/radeonsi_drv_video.so
ffmpeg verbose: libva: Found init function __vaDriverInit_1_13
ffmpeg verbose: libva: va_openDriver() returns 0
ffmpeg verbose: Initialised VAAPI connection: version 1.13
ffmpeg verbose: VAAPI driver: Mesa Gallium driver 21.3.7 for AMD Radeon RX 6700 XT (NAVY_FLOUNDER, DRM 3.44.0, 5.16.15-zen1-1-zen, LLVM 13.0.1).
ffmpeg verbose: Driver not found in known nonstandard list, using standard behaviour.
ffmpeg verbose: Input surface format is nv12.
ffmpeg verbose: Using VAAPI profile VAProfileH264Main (6).
ffmpeg verbose: Using VAAPI entrypoint VAEntrypointEncSlice (6).
ffmpeg verbose: Using VAAPI render target format YUV420 (0x1).
ffmpeg verbose: RC mode: VBR.
ffmpeg verbose: RC target: 83% of 36000000 bps over 1000 ms.
ffmpeg verbose: RC buffer: 36000000 bits, initial fullness 27000000 bits.
ffmpeg verbose: RC framerate: 239/8 (29.88 fps).
ffmpeg verbose: Using intra and P-frames (supported references: 1 / 0).
ffmpeg warning: Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
ffmpeg verbose: Using level 4.1.
CLIENT: Got video sequence 7267, expected 7265, sending lost data notification
CLIENT: Marking frame 7270 complete with status k_EStreamFrameResultDroppedReset
CLIENT: Marking frame 7271 complete with status k_EStreamFrameResultDroppedReset
CLIENT: Marking frame 7272 complete with status k_EStreamFrameResultDroppedReset
CLIENT: Marking frame 7273 complete with status k_EStreamFrameResultDroppedReset
CLIENT: Marking frame 7274 complete with status k_EStreamFrameResultDroppedReset
CLIENT: Marking frame 7275 complete with status k_EStreamFrameResultDroppedReset
Setting target framerate: 14.94 [timing: game 3.12, capture 1.92, convert 23.60, encode 10.37, network 13.52, decode 3.03, display 11.68]
CLIENT: Targeting 29.88 FPS
CLIENT: Got control packet k_EStreamControlSetTargetFramerate
ffmpeg verbose: libva: VA-API version 1.13.0
ffmpeg verbose: libva: Trying to open /usr/lib32/dri/radeonsi_drv_video.so
ffmpeg verbose: libva: Found init function __vaDriverInit_1_13
ffmpeg verbose: libva: va_openDriver() returns 0
ffmpeg verbose: Initialised VAAPI connection: version 1.13
ffmpeg verbose: VAAPI driver: Mesa Gallium driver 21.3.7 for AMD Radeon RX 6700 XT (NAVY_FLOUNDER, DRM 3.44.0, 5.16.15-zen1-1-zen, LLVM 13.0.1).
And again note, it's not a 'resource' issue. I was trying this on simple vulkan games, vs 'expensive' oGL games. oGL games could stream at 60fps just fine. Vulkan games switched modes to Game Vulkan RGB + libyuv + VAAPI H264, and fps tanked, and held at 30fps.
When I tab out of that, the game keeps running, and the stream goes to 60fps again after switching out of that encoder mode.
I literally think I could run a game in, e.g., gamescope, then stream the window at 60fps. I'll test that.
Edit: Gamescope test didn't exactly work due to some crashing issues with gamescope itself. However, I tried a few other games by just alt-tabbing out and letting the game run - every time it would soar from ~30fps to > 60fps. I'll try some other method to test, like launching a vulkan lutris game (e.g., stalker: anomaly), then just streaming a monitor of that [rather than the game itself] in OBS. My hypothesis is that it'd be a smoother experience than streaming the game itself, despite it being encoded twice (once in OBS; once in steam).
Edit 2: I thought I would mention that I have now tried this in X11, Wayland (-pipewire), and Wayland (-pipewire-dmabuf). They all exhibit the same behavior.
So I've also found the same. If I alt+tab off to anything else, it falls back to Desktop OGL or whatever, and framerates better CPU usage is lower and everything is great.
I really cannot understand what is going on here. And why it's still an issue 4 years later
What I've found in my case is that if I make a game use (regular, non-borderless) windowed mode and adjust the game resolution to the streaming client's resolution, things run a lot smoother, so it's definitely the scaling that's a problem (my desktop uses 1440p and my client devices usually 1080p[^1]). It's so prevalent and reproducible that it must be reasonably easy to pinpoint the problem with access to the internal streaming code, and I guess this issue board is long out of date and maybe it needs to be raised elsewhere to get the attention of valve's devs.
[^1]: worth noting that it only works in windowed mode, fullscreen is choppy regardless, and I suspect this is to do with proton's fullscreen hack, which avoids using exclusive fullscreen mode in favour of scaling the output to the full resolution. This also means that the steam streaming option to "change host machine to client resolution" seems to do nothing most of the time, on my system.
Replying to https://github.com/ValveSoftware/steam-for-linux/issues/5591#issuecomment-1073744693
I did test the scaling issue by ensuring the host and client resolutions are the same (and the game res is also the same), but I only tested using fullscreen and borderless windowed. I'll try with the bordered window next time.
Yes this reduces the load. However for some of us with older CPU's this doesn't really buy us much
Yes this reduces the load. However for some of us with older CPU's this doesn't really buy us much
I see no difference in load between the two. Scaled or not, it's locked at 30. When alt tabbed, it's over 60 fps, whether scaled or not.
I don't have an amazing CPU either. Until this Wednesday, I have an i5 6500. I'll have a Ryzen 5600x on Wednesday to further test this with.
Replying to https://github.com/ValveSoftware/steam-for-linux/issues/5591#issuecomment-1074610569
I wonder if I'm having a separate issue from you (possibly hardware related as I'm on Intel i7 8700k) - I don't get so much a hard capped FPS rate as a super stuttery mess that can fluctuate a lot, it doesn't feel like a cap.
There is definitely a difference is CPU load. I've got an i7 3930
All the colour space conversion and resize is done on CPU. Removing the resize allows a faster frame rate due to reduction in CPU usage and pipeline latency.
The reason the frame rate is all over the place is because the pipeline is non-deterministic. It takes a different amount of time for each frame.
Lo and behold, moonlight/sunshine work absolutely fine. This is purely an encoder setting issue I think. I can stream RDR2, maxed out, at 60 fps, 1080p, no issues (and it's only 60fps due to vsync in this case). The same benchmark through the steamlink app varies between 30-40fps, at best, with a ton of stuttering. It was smooth as butter in moonlight.
And luckily, you can put moonlight on the steam link itself (huzzah). So that's a decent workaround. However - valve should really fix this issue; it's really bizarre that FPS can tank so incredibly much when using the steam link app, and only when in vulkan games.