Wayland: Emulate mouse warp using relative mouse mode

[sulix]

Several games (including Source and GoldSrc games, and Bioshock Infinite) attempt to "fake" relative mouse mode by repeatedly warping the cursor to the centre of the screen. Since mouse warping is not supported under Wayland, the viewport ends up "stuck" in a rectangular area.

Detect this case (mouse warp while the cursor is not visible), and enable relative mouse mode, which tracks the cursor position independently, and so can Warp successfully.

This is behind the SDL_HINT_VIDEO_WAYLAND_EMULATE_MOUSE_WARP hint, which is enabled by default.

Note that there is a behavioural difference, in that relative mouse mode typically doesn't take mouse acceleration into account, but the repeated-warping technique does, so mouse movement can seem very slow with this (unless the game has its own mouse acceleration option, such as in Portal 2).

[flibitijibibo]

I can definitely see this breaking some games - some games always draw a software cursor but use warping only when in-game. This is definitely possible but will require an extra picky QA pass.

[icculus]

I know we're not talking about just the mentioned titles, but it might be worth just fixing GoldSrc and Source to do this right.

[sulix]

I tried this with quite a few games — including ones which used software cursors — and nothing actually broke, as far as I could tell. The heuristic of waiting until there's at least one warp before enabling it works pretty well, and running a software cursor with relative mode enabled is not actually "buggy". The big downside is that you lose mouse acceleration, which can definitely feel weird if you're not expecting it.

It's worth noting that The Witcher 2 uses a software mouse cursor, but it breaks if SDL_WarpMouse() doesn't work, so this actually fixes some software cursors, too…

That being said, if this does turn out to break things, and we can fix the high-profile Source engine games, then it'd probably make more sense to have this off by default.

[flibitijibibo]

I think the way to fix it is to have the detection work both ways - a lot of warping means it wants relative, but once the application stops it's probably in a "menu" state, so we can flip it back after a few (or less?) frames of no warping. I think the winewayland branch by Collabora has been exploring this as well, might be worth taking an idea or two from them...

[sulix]

I quickly hacked together some code to disable this if there's been no warp in the last 4 frames.

None of the games I tested were affected though, so I'm not actually sure if this works:

• The Witcher 2 uses a fake "relative mouse mode" by warping each frame, even for its software cursor (so it warps every frame even in the menus).
• Portal 2 shows the mouse cursor in the menu, which disables relative mouse mode immediately.
• Life Is Strange uses a software cursor, but uses SDL's relative mouse mode correctly, rather than warping each frame.
• Stronghold 3 only warps when using the middle mouse button to rotate the screen, and re-enables the mouse cursor afterwards.

[flibitijibibo]

It's been a while but I believe Murder Miners and Proteus are two examples of doing both... maybe Cryptark as well? Superliminal does a little bit of manual ShowCursor work too, if you pass -hwcursor as a launch option.

[sulix]

Cheers: I've given Proteus a bit of a try, and realised that there were a few broken things in that previous version. Proteus indeed did end up still being stuck in relative mouse mode when returning to the main menu (though the menu is perfectly usable even with relative mouse mode — it took me a few goes to notice the difference).

[There's also an unrelated issue with Proteus not resizing to fill the screen properly on startup, needing to be minimized and restored, but that might be related to the nvidia EGL hack or similar. I'll look into it when I find the time.]

Unfortunately, it looks like Portal 2 doesn't warp the mouse every frame, but only when it needs to (presumably when it gets far enough from the centre of the screen), which trips the "unlock if we haven't warped in 5 frames" code pretty badly. Sometimes, during normal play, it'll take several seconds between warps, at which point we disable relative mouse mode and get a great big movement event to synchronise with the real cursor position, making you suddenly face the opposite direction in the game. Even in the best case, movement is horribly, unplayably choppy.

I'll try to have another look at what XWayland and wine-wayland are doing to see if their heuristics are any better (though I suspect they're doing something more complicated than just enabling/disabling relative mode).

[flibitijibibo]

Can confirm Proteus' viewport issues are application issues - will probably fix that over the weekend since that game has other problems (as I discovered last night, yech).

Totally forgot about Source's weird warp behavior... I wonder if we should just disable non-raw input in those games (since they're already updating them for Deck anyway).

[sulix]

I reverted the code to disable relative mode after a certain amount of time without a warp, and instead now have SDL_HINT_VIDEO_WAYLAND_EMULATE_MOUSE_WARP default to 0 if the application ever enables relative mouse mode. This will obviously break in a few edge cases (most notably anything which warps in the menu before ever enabling relative mouse mode, but there are other cases, too), but it seems to work in practice on everything I've tried so far. And the hint can force it both always on, or always off.

I've also rebased it on top of the latest main, largely to test it against the fix for #5569 (which might have contributed to some of the issues I was seeing). Everything still seems to work, but it's possible the automatic disabling of relative mode from the previous revision would've been slightly less broken with that applied.

[AJenbo]

DevilutionX depends on this when opening and closing the inventory and other panels

[slouken]

@sulix, can you rebase this on current main and give it a whirl? I'm ready to merge it.

[sulix]

Rebased onto current main (and the hint stuff updated). It worked on the few games I tried it on (notably Portal 2), but I'll test it some more over the weekend. (I think there's an issue with games which use relative mode, when the hint is forced on, but I haven't got any games which need that, IIRC.)

[jwrdegoede]

Any chance we can also get this added to 2.24.x?

I've been using this locally for a while to fix issues with some games in dosbox-staging (which uses SDL2 unlike the original dosbox). An alternative workaround for this is to switch to X11 mode, which I now is the default upstream. But at least for Fedora where the SDL2 default is still set to Wayland having this patch in 2.24.x would be helpful.

[jwrdegoede]

I've also asked Fedora to consider carrying this as a downstream patch: https://bugzilla.redhat.com/show_bug.cgi?id=2137528

[flibitijibibo]

I think 2.26 is due next week anyhow, so if not 2.24 the next feature release is coming up really soon to replace it.

[sulix]

Yeah: I'd be wary of including this as-is in a stable update: it's a real behaviour change. It'd probably be fine to backport it but I'd probably make it disabled by default just in case.

Otherwise, waiting for 2.26 seems safer to me.

[jwrdegoede]

I did not realize 2.26 will be released soon, in that case waiting for 2.26 is fine.