Support for pure-Wayland environments

[kflak]

Environment

• SuperCollider version: sclang 3.10.2
• Operating system: Arch Linux, 4.19.59-rt24-1-rt-lts
• Other details (Qt version, audio driver, etc.): QT5, Intel, Wayland (swaywm)

Steps to reproduce

{ SinOsc.ar(MouseY.kr(500, 10000, 1).poll, 0, 0.1) }.play;

Expected vs. actual behavior

Expected: value updates as I move the mouse. Actual: no change whatsoever.

Summary

SuperCollider's X11-specific code needs to be ported/reimplemented/abstracted for Wayland, otherwise it will not work on pure-Wayland systems like swaywm (sway). The following files, at least, need to change:

QtCollider/CMakeLists.txt
QtCollider/QcApplication.cpp
QtCollider/interface.cpp
QtCollider/QWidgetProxy.cpp
QtCollider/hacks/hacks_x11.cpp
QtCollider/hacks/hacks_x11.hpp
server/plugins/CMakeLists.txt
server/plugins/UIUGens.cpp
editors/sc-ide/CMakeLists.txt
editors/sc-ide/widgets/code_editor/editor.cpp

See also this guide for porting Qt applications to Wayland

[nhthn]

hi kenneth, thanks for the report. its possible that SC was built with -DNO_X11 which would cause these ugens to stop working. did you build SC yourself or get it precompiled from a package manager?

[patrickdupuis]

I can confirm using Sway from the Fedora package repositories. We likely need to start making SCIDE work on pure-Wayland environments like Sway.

[nhthn]

welp, just embarrassed myself by having no idea what wayland is haha

now that i've googled it -- yeah the problem is that the UIUGens use X intrinsics so they can't work with wayland. that's definitely worth fixing. are sclang or scide impacted at all?

[mossheim]

are sclang or scide impacted at all?

As far as i understand, no; this is just a change for UGen code.

[nhthn]

right but are there any known issues for sclang/scide on wayland?

[mossheim]

ohh. no idea :)

[patrickdupuis]

I don't understand this stuff very well myself. But, I can say that I've been using some combination of Gnome3 + Wayland since it became the default on Fedora a couple years ago. I haven seen any issues. I can definitely use MouseX. I'm guessing distros who use Wayland make sure that there's some sort of X compatibility available for applications who need it.

[kflak]

Hey guys, a little late to the party here... Anyway, I installed SC precompiled from the Arch repositories. The PKGBUILD uses these flags:

cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
           -DCMAKE_BUILD_TYPE=Release \
           -DLIBSCSYNTH=ON \
           -DFORTIFY=ON \
           -DSYSTEM_YAMLCPP=ON \
           -DSYSTEM_BOOST=ON \
           -DSC_VIM=OFF

I don't know too much about input handling in X vs Wayland, but it seems the input layers are quite separate. Normally I don't have any problems running X applications on Sway, this is taken care of by the xwayland compatibility layer.

[nhthn]

cool, thanks for the info. the next step would be to try building SC yourself to see if this is an issue with SC or with the arch package. if it's a problem on the packaging end then this is a @dvzrv question. (sorry for pinging you so much this weekend dave)

[sensestage]

Just for the record, other affected UGens would be MouseButton and KeyState. They use the same x-interface. See server/plugins/UIUGens.cpp/ lines 98 to 140.

[sensestage]

And looking for X is done through autodetection in the CMake files, lines 283 to 294 in the same directory:

if (CMAKE_SYSTEM_NAME MATCHES "Linux" OR CMAKE_SYSTEM_NAME MATCHES "FreeBSD")
	find_package(X11 REQUIRED)

	if (NOT X11_Xt_FOUND)
		message(FATAL_ERROR "libxt not found")
	endif()

	include_directories(${X11_Xt_INCLUDE_PATH})

	foreach (ugen ${uiugens})
		target_link_libraries(${ugen} ${X11_X11_LIB})
	endforeach()

[MaxWorgan]

As a workaround you will notice that starting scide with QT_QPA_PLATFORM=xcb will force it to use Xwayland, which in turn allows the use of the X11 libraries and capture mouse events.

It does depend on what is 'focused' as to whether the mouse events get captured, even without above line, if you switch to an Xwayland enabled app, the mouse events will suddenly start being captured.

[salkin-mada]

On Debian 10 (Sway wm 1.4 build via sid package meson)

@snappizz As mentioned by @patrickdupuis there is no problems with sclang/scide on Wayland. As long as the distro/wm used supports Xwayland for backwards compatibility. Which Sway does.

• SuperCollider from 3.10.2 through 3.10.4-rc1 to 3.11.0 (using same build flags as OP's PKGBUILD) ** SCIDE is Xwayland by default, using xeyes spills the beans. And Mouse{X,Y} works. ** Kitty with SCNvim runs natively under Wayland. (using this combo I suspect the OP to be guilty off) :-) The Mouse{X,Y} does NOT work. And in this situation doing a Window.new("hacky_mouse", Rect(555,555,555,555)).front; will make the Mouse{X,Y} work

@kflak Or did you use SCIDE for this test?

@patrickdupuis I really agree (running SCIDE and all SuperCollider content natively on Wayland would be lovely)

We likely need to start making SCIDE work on pure-Wayland environments like Sway.

@sensestage I guess we should look for Wayland here in addition to X instead of just going directly towards the FATAL_ERROR

if (NOT X11_Xt_FOUND) message(FATAL_ERROR "libxt not found") endif()

But I am no seasoned dev. Let me know if I can assist in any way with this.

I think this is highest priority for a 3.11.1 release :) but thats just me over here on Wayland

https://github.com/swaywm/sway/wiki/Running-programs-natively-under-Wayland

[kflak]

@salkin-mada Guilty as charged: scnvim with a wayland-native terminal emulator (termite, not kitty...) Thanks for the hacky solution! I am a bit attached to my setup and only use scide when strictly necessary (i. e. teaching...)

[mossheim]

for someone who is relatively new to Linux and windows managers but wants to help -- what are the action items here? i'm confused by the discussion above.

[kflak]

@brianlheim For a bit on the different display backends on Linux: xorg and wayland

sway is a minimalistic window manager running on Wayland.

[mossheim]

@kflak thanks, but i'm looking for a clear explanation of what in our codebase needs to change.

[salkin-mada]

@brianlheim I have been poking around at channels #qt and #wayland. But I have trouble asking the correct questions :) due to my lacking knowledge of how SuperCollider uses Qt5/X11 or the SuperCollider codebase in general. But Qt people say that "if it's a Qt5 app nothing needs to be done". - But of course if direct X11 calls is being done, it is a different situation. The Wayland people say: "don’t do direct X11 calls, mostly." I was pointed towards this guideline and porting note

In the case of needing to make the application platform specific i was expecting something like:

#if HAVE_X11
    if (QX11Info::isPlatformX11()) {
        callX11SpecificImplementation();
    }
#endif
#if HAVE_WAYLAND
    if (QGuiApplication::platformName().startsWith(
            QLatin1String("wayland"), Qt::CaseInsensitive)) {
        callWaylandSpecificImplementation();
    }
#endif

But i am waaay out of my comfort zone or knowledge here. Have no real understanding of what is going on. :) Can you point me towards where in our codebase we open a connection to the X Server? If thats how it works with SuperCollider.

[salkin-mada]

@brianlheim - @sensestage was pointing to some detection in the CMake and I think what needs to be done is to make the codebase not use libXt anymore. But I am not sure of which lib to use instead hence there is no one lib what would match on Wayland what libXt does on X11. libXt's API is inherently tied to X11 I assume.

Wayland says (/NICK pg): "if you have any piece of code that uses libXt, that piece of code needs to rewritten completely, and quite possibly also re-designed." -- hmm.

Wayland does have some fundamentally different design decisions, like the lack of a global coordinate system.

'pg' says: "But SuperCollider uses Qt already. Why does it then use libXt at all?" and "any X11-specific library the app uses on its own and not abstracted by Qt is going to be a problem for a Wayland port. Because if you are making a Wayland native app, you cannot run any X11-specific code"

if (CMAKE_SYSTEM_NAME MATCHES "Linux" OR CMAKE_SYSTEM_NAME MATCHES "FreeBSD")
	find_package(X11 REQUIRED)

	if (NOT X11_Xt_FOUND)
		message(FATAL_ERROR "libxt not found")
	endif()

	include_directories(${X11_Xt_INCLUDE_PATH})

	foreach (ugen ${uiugens})
		target_link_libraries(${ugen} ${X11_X11_LIB})
	endforeach()

[mossheim]

due to my lacking knowledge of how SuperCollider uses Qt5/X11 or the SuperCollider codebase in general.

got it. we use Qt5 for the vast majority of the codebase that interacts with windowing managers. X API is used sparingly in QtCollider (Qt extensions for the sclang programming language), IDE, and in the UIUGens (MouseX, MouseY, KeyState). this is the list of files that include an X11 header:

QtCollider/CMakeLists.txt
QtCollider/QcApplication.cpp
QtCollider/interface.cpp
QtCollider/QWidgetProxy.cpp
QtCollider/hacks/hacks_x11.cpp
QtCollider/hacks/hacks_x11.hpp
server/plugins/CMakeLists.txt
server/plugins/UIUGens.cpp
editors/sc-ide/CMakeLists.txt
editors/sc-ide/widgets/code_editor/editor.cpp

am i correct in understanding that our goal now essentially is to take any X11-specific code in these files and rewrite it in a Wayland or a suitable abstraction layer equivalent?

[mossheim]

I was pointed towards this guideline and porting note

this is very useful, thank you!

[salkin-mada]

am i correct in understanding that our goal now essentially is to take any X11-specific code in these files and rewrite it in a Wayland or a suitable abstraction layer equivalent?

that is my understanding at least.

[mossheim]

great. I've updated the issue title and description to reflect this discussion. hoping we can solve this by 3.11.2 at the latest, but that will depend on how difficult it is to port.

[mossheim]

working on this now. my plan is to start with porting the X11-specific code in UIUGens followed by the IDE and QtCollider, and then begin tackling any obvious bugs in our Qt code that break on Wayland. i've installed sway on my Arch Linux box to check my work.

my thinking on the build system support is:

• configuration options for both Wayland and X11
• if both are enabled, which to use is chosen dynamically at runtime, preferring X since that is the more stable code
• probably, environment-specific code should print a warning while initializing if it is being run in an environment that it wasn't compiled to support

i would prefer to avoid 'automagically' choosing libraries based on what is available on the user's system, but i am not sure what a proper default would be. wayland and x11 seem available enough and small enough that requiring their libraries and qt5-wayland by default seems reasonable and practical. note that a silent clever choice in the build system often leads to headaches later, by which i mean that rerunning CMake and/or installing a library takes much less time and frustration that rebuilding or reinstalling the entire project. thoughts welcome.

[salkin-mada]

Sounds like a plan. Good idea to favor/prefer X in a situation where both is enabled. As an example as of now on Debian Stable 'qt5-default' is v5.11.3 and it is my understanding and experience this is not a new enough version to support qt apps natively under Waland via 'qtwayland5'. But I may be mistaken. (this will not be a problem for people using Debian Sid or Arch). Nevertheless in a situation where a setup is running some old version of Qt and has both Wayland and X enabled, while SuperCollider is favoring Wayland it could turn out to be troublesome. Preferring X sounds like a safe choice.

[mossheim]

after some more research, i've realized this will require an alternate approach for UIUGens. wayland has no way of grabbing global keyboard and mouse state by design. getting input events requires window focus. scsynth of course has no windows and we don't want to create one let alone require it to have focus in order to use these UGens. IIUC there are at least four options worth considering at this point, but i am a novice to this domain so there could certainly be more possibilities.

1. use functions from the linux/input.h header directly, which would require special permissions with udev(?). i am clueless here but it seems like this should be possible. obviously with this approach we do not want users running scsynth/sclang as root, nor do we want to give every user application direct access to device event streams.
2. use Qt. i don't know if that makes things easier or if it even works here for that matter, but if it works there would at least be a consistent abstraction layer.
3. use some other higher-level library. i looked into a few. KWayland depends on Qt, and GLFW, GLEW, and Gtk only allow input events to come through when the window has focus, so none of those are viable AFAICT.
4. document the issue and wait until a better solution comes along. this is what i'm in favor of currently, because i've exhausted myself on obvious solutions.

for now, you can use XWayland with sway.

as for sclang/scide support for Wayland, i am inclined to deprioritize that if these problems persist and XWayland is a workaround in the meanwhile.

[salkin-mada]

@brianlheim thanks for looking into this! yes it seems that by design in Wayland a surface only knows the cursor position when the cursor intersects with a non-occluded section of it.. which is useless(troublesome) for scsynth and the UIUgens. The Wayland devs say that it will never be possible, it's a security and isolation issue. It's a bad pattern to rely on this global input state mentality. If Wayland was to support this it would mean letting every application on a system be a keylogger if it wanted to, which is the case under X11/Win/MacOs :=)

Indeed go for 4. And for the users who wants the highly secure compositor, which is Wayland, they need to make a small Window to get mouse position and button states into scsynth(UIUgens)

[mossheim]

yes i'm aware what wayland devs say. seems like (1) is still an option if someone is willing to look into it.

[eleses]

I suggest this issue be retitled to be about scsynth only. There are three more problems I've noticed with scide on wayland systems, e.g. on Fedora 32, #5035 #5036 #5037 . Apparently since Fedora 31 Qt uses wayland directly on Fedora, which causes scide problems too. And yeah, people complained about the change breaking other apps so... Red Hat's answer was to "Blacklist some applications depending on X11 API to be used on Wayland". But nobody complained about SC to them so... SC(ide) is not on the fallback list.

---

As for actually making the mouse server UGen work, I suspect the only solution is to create a fullscreen but transparent window, which still possible in wayland.

wl_shell_surface::set_fullscreen - make the surface a fullscreen surface

According to the comments in the code, Wayland always supports ARGB, so transparency should not be an issue (even though the docs on freedesktop.org don't say this so clearly.) Frankly a fullscreen transparent window already sounds like a security risk, so maybe it's not actually allowed... worth testing though. If that's possible, then you can

wl_seat::get_pointer - return pointer object

to access the "mouse" within that surface.

The real problem is going to be interacting with other windows while this fullscreen thing is displayed. There is no way to forward the requests, as far as I can tell. So maybe make it so it ends (disappears) if it's clicked. Mouse Ugen is not really used in production in my own experience, and for help/demo code, ending on click would not be a big deal.

Actually, I see there's also a MouseButton UGen which woould not be happy with that. Probably having a special key then to exit (e.g. ESC) might be best. And instead of a fully transparent window, it could display this key in some place (corner or top).

---

use functions from the linux/input.h header directly, which would require special permissions with udev(?). i am clueless here but it seems like this should be possible

I actually thought about that too (in fact using libinput would be the best "direct" approach for getting input on linux nowadays), but the problem is that you'd still not have an absolute position on a screen surface. For a mouse (and even for a touchpad) you can only get relative movements from libinput. Only for touchscreens you can get absolute ones.

use Qt. i don't know if that makes things easier or if it even works here for that matter, but if it works there would at least be a consistent abstraction layer.

Qt wayland can't seem to do anything that wayland won't let you do. (And likewise for any other library on top of wayland.) And QtWayland probably has bugs of its own... (see the above 3 ones). And I'm not sure you'd really want to link scsynth to Qt... Acually scsynht itself is not even linked against X11. Only the mouse/kbd ugens UIUgens.so) is linked against libxcb, libX11 etc. (Interestingly scsynth is linked to libdbus and libsystemd, so I'm guessing some stuff won't work with a different init.)

document the issue and wait until a better solution comes along

Yeah, it seem the only easy option right now. Given the issues with scide running on (Qt) wayland directly (i.e. bypassing Xwayland), I'd say probably full support for wayland in SC will take a fair bit of work... and for the mouse/kbd inputs on the server, I really don't see an alternative to spawning a virtual surface on wayland.

[eleses]

I see that Red Hat has proposed a number of unstable/experimental extension protocols for Wayland. One of these would allow keyboard input to non-focused window... but restricting it to Xwayland.

I could not find anything proposed for absolute pointer position (on the screen), but they did propose one for constrained pointer movements and one for tablets (and one for gestures).

So yeah, Wayland is still a work in progress even protocols-wise.