In-home Streaming + Wayland

[Mushoz]

Your system information

• Steam client version (build number or date): February 18th build
• Distribution (e.g. Ubuntu): Arch 5.0.2 Kernel
• Opted into Steam client beta?: No
• Have you checked for system updates?: Yes, system is fully up-to-date

Please describe your issue in as much detail as possible:

Whenever I run my desktop with Wayland, and connect my steamlink to my Desktop, the screen on the TV will be rubbish with every color of the rainbow all over the screen. The screencapture of the desktop is clearly not going as planned. Actions still work fine, and I can navigate via the controller connected to the steamlink by simply watching what I am doing on my computer screen. Once I launch a game, the corrupted mess disappears and I can properly see the game being streamed to the steamlink.

Steps for reproducing this issue:

1. Use Wayland (I am using Gnome on Wayland in case that matters)
2. Start steam
3. Start steamlink
4. Connect to desktop

What happens: I see a corrupted mess on my TV

What should happen: I should be seeing steam big-picture mode, as my computer display is showing.

Workaround: Use gnome on Xorg.

[MayeulC]

I am affected as well. Steam Remote play doesn't work, neither does remote play together. Input and audio works perfectly.

The computers used for testing were:

• Archlinux desktop (AMD R9 Fury, Mesa, RADV), native steam (and flatpak)
• Archlinux laptop (Intel integrated graphics), via flatpak

Clients tested:

• Android application, LAN/LTE
• laptop with sway (with and without hardware decoding)
• desktop with sway (with and without hardware decoding)

Servers tested:

• :x: desktop with sway (with flatpak, without, with and without HW encoding)
• :x: laptop with sway (with and without HW encoding)
• :x: desktop with plasma wayland (with HW encoding)
• :heavy_check_mark: desktop with i3 (with HW encoding)
• :heavy_check_mark: desktop with plasma on X11 (with HW encoding)

I am thus reasonably certain that the issue is with Wayland or XWayland (I would need to test plasma without HW encoding to be sure there are not two separate symptoms here). Screen sharing from applications like Firefox do not work on Wayland either. A possible future solution might leverage pipewire for capture.

I haven't seen anything on the client besides corruption (like uninitialized memory, with a few textures from former apps), even after a game starts. I used simple games for my tests (FTL: Faster than light, Crypt of the necrodancer), and only tried 3D games like Portal 2 on X11 to confirm it still worked. Do 3D games use a different capture method? (Vulkan layer, OpenGL lib injection). Which ones should I try?

[ghost]

Same issue on Plasma Wayland (tested on Arch Linux). I assume Steam would have to ask pipewire to stream the desktop.

[tgunnoe]

Same issue on Sway

[PMaynard]

Same on Fedora 32, with RPMFusion steam, and Sway. Audio works. But no visual on the steam link device iPhone current iOS.

[sinedoOo]

Same issue on Fedora 32 + Gnome wayland

[PMaynard]

anyone tried the flatpak version?

[fuzunspm]

same issue on Arch Linux swaywm/wayland + iPad setup

[MayeulC]

anyone tried the flatpak version?

@PMaynard: Re-read my comment, I am using the flatpak version.

This is pretty much a known issue for every wayland environment at that point, so I don't think there's much point in adding pages of comments that everyone has the same issue, and it would be more productive for everyone to upvote the issue.

Valve seems to be working on a wayland compositor, I believe the capture functionality could be integrated into it, and then have the compositor act as a client to also display stuff on the screen (if needed). Not sure if that's the way they want to go forward with, but that's what I would do. Pipewire and waypipe are other, more general options.

[yokem55]

Playing around with this a bit more - in a sway session at least it seems that only the big picture interface has trouble being streamed. Starting a game while looking at the normal monitor with the keyboard launches the game, and the game will then appear over the steam link (android app or standalone device) properly. This is only a partial work around, but it does mean xwayland has what it needs to capture and stream the game.

[MayeulC]

@yokem55, that's interesting, it never worked for me. Neither steam nor browsers are able to capture other Xwayland windows, only displaying a black background. @emersion hinted that it should work already, so I am not sure what's going on, I will re-test soon. Some of it could be due to multi-monitor specificities.

[awmath]

@emersion hinted that it should work already, so I am not sure what's going on

I think there has been a missunderstanding as @emersion was talking about Steam Play, which works fine on wayland, whereas this topic is about Steam Remote Play, which doesn't.

[maweki]

@emersion hinted that it should work already, so I am not sure what's going on

I think there has been a missunderstanding as @emersion was talking about Steam Play, which works fine on wayland, whereas this topic is about Steam Remote Play, which doesn't.

I think you misunderstood. For me as well, steam link/remote play works on wayland for a game but the big picture mode window is just black with a few artifacts.

That's exactly what I'm seeing. So streaming demonstrably works. Just unusable if you can't select a game.

[maweki]

I don't want to create additional noise, but to document what I think we're all talking about.

1 My Steam Link Android Phone screen before connecting

2 Pressing Play on the phone, trying to stream big picture mode

3 Quick-Selecting dicey dungeons below, skipping big picture streaming

And as OP posted, even for the arrifacted big picture streaming, commands from the phone get relayed just fine. You just can't see it.

[awmath]

@maweki True, I should have been more specific. The Problem is not streaming the games on Wayland via Steam Remote Play, but anything desktop related. It still holds true, that @emersion was talking about Steam Play, which has nothing to do with Steam Remote Play. Furthermore, at least for me it is not the Big Picture mode itself which is the problem. You can go ahead and in the Steam Link settings choose "Launch Mode: Desktop". This won't bring up the Big Picture Mode but still won't stream anything useful until you start a game.

[awmath]

I finally found a workaround! The problem seems to be the wayland compatibility of steam. This seems counter-intuitive at first, but seemingly all screen capture software has trouble with wayland (just do a quick search for wayland screen capture black screen on the net) So steam has to run under x11 to support remote play. Luckily we have a way of using x11 applications under wayland with Xwayland. Now we "just" have to force steam to run under Xwayland

First we start a Xwayland server with a seperate display Xwayland :11. The display number should be any display number which is currently unused. Then start steam with DISPLAY=:11 steam. Done.

You can fiddle with the Xwayland settings until you are satisfied Now just to make it a little bit more comfortable...

[maweki]

I finally found a workaround! First we start a Xwayland server with a seperate display Xwayland :11. The display number should be any display number which is currently unused. Then start steam with DISPLAY=:11 steam. Done.

So what you're basically saying is, while Steam or Linux (who is responsible for that?) seems to run most games in XWayland already, the steam client proper does not and is therefore not capturable? This would imply that games that work with and use wayland would not be streamable either, yeah?

[awmath]

So what you're basically saying is, while Steam or Linux (who is responsible for that?) seems to run most games in XWayland already, the steam client proper does not and is therefore not capturable? This would imply that games that work with and use wayland would not be streamable either, yeah?

That's correct. In fact that's how I stumbled upon this "solution". I tried adding non-steam applications to see which could be captured and which not. And even some steam games couldn't be captured (Wasteland 2 for example). By default all applications should start under wayland and only legacy applications shouldn't. By running steam inside the Xwayland server, steam will start all other applications in this instance as well. So all games should be streamable from there on.

Edit: One more observation. Starting Xwayland with the -rootless argument (which would be desirable) won't work. Which leads me to believe that maybe it's not that steam isn't running in a Xwayland session, but rather the default Xwayland implementations in Gnome/Sway/"you name it" all use the rootless Xwayland path. While this is definitly the right choice I wonder if there is a way around this.

Edit2: For gnome I can confirm that Xwayland runs as rootless which causes the issue with remote play. So one fix could be to disable Xwayland for Gnome alltogether by adding --no-x11 to the gnome-shell command in the systemd service file, and autostarting a Xwayland server with a custom command and without -rootless.

Also for all wlroots based wayland compositors (sway, cage, gamescope, ... ) this is a bigger Problem as it is hardcoded in https://github.com/swaywm/wlroots/blob/6c08fe979615ac88648eb91c87ea4c41fa1d7bdf/xwayland/server.c#L65. Though there has been a merged pull request in october which could help here. If we create a new file with the content:

#!/bin/bash
Xwayland ${@##-rootless}

make it executable and pass it to the compositor via WLR_XWAYLAND=... the Xwayland server should be started without the rootless argument. Given the used compositor uses the required version of wlroots (>0.12.0)

[Tzigamm]

Any progress on this? Wayland is finding its way into more and more systems with Fedora, Ubuntu and others enabling it by default. Some players (me included) switch to wayland for gamer-oriented features such as a better multi-munitor support, better high refresh rate support natively and native HiDPI options, as of now I cannot stream any game to my Steam Link nor to any of my friends through Remote Play Together without switching first to X11 (which is really not an option in my case). Now that multiple Wayland streaming alternatives exists (most browsers support screen sharing on wayland, OBS works fine too) I hope Valve can do something about it before too many of us are stuck without a very cool and useful feature

[michaelnew]

So here's what I think is happening, plus 2/3 of a workaround (maybe someone else can figure out the rest of this):

Most games run in an Xwayland window, and Remote Play is able to stream the contents of that window just fine. Anything that's running in a native Wayland window doesn't work (DOOM Eternal, for example), because Remote Play isn't using any capture methods that work on Wayland.

Even though Steam itself runs in Xwayland (it's definitely not Wayland native), it doesn't try and stream the contents of that window. Instead it tries to capture the entire desktop, which won't work if you're using Wayland. It's probably a sensible choice since a lot of games will try and show little launchers or popups or whatever and you'd want to be able to click on them or do whatever you need to do to get the game to launch. But regardless; it breaks streaming for Steam itself.

If you do something like this: Xephyr :1 -screen 1920x1080 & DISPLAY=:1 steam it will put steam into it's own little X11 box, and when it streams the "desktop" it's really just streaming the contents of that X instance. The problem is that games try to launch inside of that X instance, and at least the games that I tried didn't work (missing Vulkan libs or somesuch). But you can make games launch outside of Steam's little sandox by setting the launch options to DISPLAY=:0 -- %command%.

However... Remote Play doesn't switch to streaming the contents of that Xwayland window. It just sits there streaming the Big Picture interface. So if there's a way to convince Steam to stream that window I think we'd be in business.

Obviously the real solution would be for Steam to be able to capture the desktop on Wayland, probably with xdg-desktop-portal. OBS did this recently. There's a plugin here that was merged into OBS proper. So it's definitely doable now.

[minikN]

Is this being actively worked on by Valve?

[maweki]

Is this being actively worked on by Valve?

I imagine it must. At least the odd guy. The plan for SteamOS and the new hardware can not be to stay on X for the rest of eternity. It would be unusual for Valve to drop the ball on this long-term architecture stuff.

[cinerea0]

Has anyone come up with any other solutions for this problem? Neither the Xephyr not Xwayland solutions listed above have worked for me.

[kode54]

Gamescope now has a PipeWire stream, which was just implemented less than a month ago. The issue is linked above.

[Mushoz]

@kode54 What exactly do you mean? What issue is linked above? And what exactly does Gamescope have to do with in-home streaming not working properly under Wayland?

[minikN]

@Mushoz If I understand correctly, PipeWire allows for screen capture under Wayland, which seems to be the capability missing currently. PipeWire is an independent library Steam could also use to remedy the problem of screen capture under wayland.

Issue: https://github.com/Plagman/gamescope/issues/130#issuecomment-727611236 PR: https://github.com/Plagman/gamescope/pull/219

[etcinit]

The latest Steam client beta added PipeWire support:

Enabled pipewire desktop capture by default on Linux, pass -nopipewire on the command line to disable it

https://steamcommunity.com/groups/SteamClientBeta/announcements/detail/2872724078255992088

---

I gave it a quick try on Fedora 34 (Gnome 40, Wayland, Intel UHD + Nvidia 470.63.01):

• Steam requests desktop capture as soon as it launches (e.g. In Gnome, the "Share Screen" modal/portal is shown, asking for confirmation.)
  • Note: I think an improvement could be to request this when streaming is about to start instead of when the client launches.
• I'm able to stream games as long as I disable the Steam in-game overlay.
  • Streaming seems to freeze as soon as the overlay notifications are shown.
  • While frozen, streaming does resume as soon as the game is closed.
  • Disabling the overlay seems to resolve the issue.
  • Games tried: Shadow of the Tomb Raider, Parkitect (both in 3840x2160)
• Streaming my entire desktop does seem to work well.
• Using Big Picture mode while streaming crashes after a couple of minutes.
  • Crash: src/steamUI/gamestream/desktopstreamthread.cpp (151) : Driver deadlock in hardware accelerated desktop capture, aborting
  • I worked around this by exiting Big Picture as soon as I start streaming.

[Willdrick]

  * Streaming seems to freeze as soon as the overlay notifications are shown.

That's not a streaming issue, it's steam client itself being half-borked under wayland. Notifications freeze and stutter, chat becomes unresponsive whenever there's an animation or multiple steam windows open (i.e. friends + library + chat)

It's been like this for a long time, and the only way to get a fluid steam experience (that also breaks launching most games) is passing env SDL_VIDEODRIVER=wayland steam

https://github.com/ValveSoftware/steam-for-linux/issues/4924 https://github.com/ValveSoftware/steam-for-linux/issues/7245 https://github.com/ValveSoftware/steam-for-linux/issues/8020 https://github.com/ValveSoftware/steam-for-linux/issues/6500 etc..

[ljrk0]

The latest Steam client beta added PipeWire support:

Enabled pipewire desktop capture by default on Linux, pass -nopipewire on the command line to disable it

https://steamcommunity.com/groups/SteamClientBeta/announcements/detail/2872724078255992088

I gave it a quick try on Fedora 34 (Gnome 40, Wayland, Intel UHD + Nvidia 470.63.01):

* Steam requests desktop capture as soon as it launches (e.g. In Gnome, the "Share Screen" modal/portal is shown, asking for confirmation.)
  
  * Note: I think an improvement could be to request this when streaming is about to start instead of when the client launches.

This is tracked in https://github.com/ValveSoftware/steam-for-linux/issues/8098

[...] * Streaming my entire desktop does seem to work well.

For me streaming Steam doesn't work, I just get a black screen -- I'm using Flatpak'ed steam though. I've added permissions to access pipewire though:

$ flatpak --user override --filesystem=xdg-run/pipewire-0 com.valvesoftware.Steam

If I change to a non-steam window, this works however. ~~Switching back to big picture suddenly makes things work. Weird.~~

Yep, recording its own window fails:

Changing record window: 0x1a0002c
SynchronizeClientState(): setting activity to k_EStreamActivityIdle: Steam Controller Configs - Big Picture
Bitrate adapter: state = 2, minRTT = 1, maxRTT = 102, gain = 0.75, bitrate = 2500
SynchronizeClientState(): setting cursor hidden
OnFocusWindowChanged to window type: k_nGameIDControllerConfigs_BigPicture, AppID 413090
Loaded Config for Local Selection Path for App ID 413090, Controller 4: /home/leonard/.local/share/Steam/steamapps/workshop/content/241100/1873878621/1322320470301167556_legacy.bin
CLIENT: Got control packet k_EStreamControlSetActivity
CLIENT: Got control packet k_EStreamControlHideCursor

W/o big picture, everything works.

UPDATE: After starting a game outside of BP, closing it, and starting BP again, even that works. I need to fiddle around some more.

[major-mayer]

Using Big Picture mode while streaming crashes after a couple of minutes.

* Crash: `src/steamUI/gamestream/desktopstreamthread.cpp (151) : Driver deadlock in hardware accelerated desktop capture, aborting`

* I worked around this by exiting Big Picture as soon as I start streaming.

I got exactly the same issue using Gnome/ Wayland. To be more specific it only crashes when i start a game using Steam Big Picture. I uploaded my error log here: https://pastebin.com/UrBCxAGs The interesting part should be at the end.

While desktop streaming seems to work okay, starting a game directly from Stream (no Big Picture) does not really work for me. I only get a black screen on my Steam Link App. -> Shadow of the Tomb Raider is interesting tho: The launcher works and gets transmitted to my phone, but the game itself doesn't work and Steam actually crashed

Streaming the desktop shows not the whole desktop, but that could result from me using fractional scaling.

[EuanFH]

Other than big picture crashing. I can't get streaming to fully load on any device. all devices get audio. you can see the cursor on steam client machine but the cursor doesn't move. it seems to be the cursor of the remote system. You don't get any video on mobile (launching the game first to avoid crashing).

This is on a wlr based compositor but from the comment above it seems to be effecting gnome based ones too.

Has anyone managed to get this semi working on wayland and got any tips?