moonlight-qt
moonlight-qt copied to clipboard
moonlight-stream
Can't input ~ via keyboard
Describe the bug I'm trying to use moonlight as a remote desktop solution. Using a Linux shell is essential. While trying it out, I wasn't able to type in ~ on the keyboard.
Steps to reproduce Using a Linux Server and windows portable client and try to type ~ in a shell
Affected games Is a Linux she'll considered a game?
Other Moonlight clients
- Does the issue occur when using Moonlight on iOS or Android?
Moonlight settings (please complete the following information) only switched to full screen
Client PC details (please complete the following information)
- OS: Windows 10 portable Version of moonlight
- Moonlight Version: newest
- GPU: Mobile Nvidia Quadro t100
Server PC details (please complete the following information)
- OS: Ubuntu 22.04.3
- GeForce Experience version: from repo using Sunshine
- Nvidia GPU driver: current driver from repo
Additional context Keyboard layout is German. ~ works on both systems without problems. Is that a problem of the portable Version?
For everyone who has a similar problem, I have two keyboard settings in windows German and US. When German is active, the tilde is not working. When I switch to US it works.
The keyboard has German layout. It's strange but works. I need to check if it is different with only one layout active
Did you select the German layout in both Windows (client system) and Ubuntu (server system)?
Did you select the German layout in both Windows (client system) and Ubuntu (server system)?
I did but it behaved like a us keyboard. Changing to US keyboard on the client made it work correctly. I have to try and only have one layout active. I don't know why this happens.
I don't think you're getting what I'm saying. Can you please send a screenshot of this part of your settings in Ubuntu?
I don't think you're getting what I'm saying. Can you please send a screenshot of this part of your settings in Ubuntu?
Are you referring to the input source switch? My window looks like your except in the first paragraph it's "German" instead of "Italian". I'm currently not at the same place as the Linux host so I can make a screenshot tomorrow if needed.
No it's not necessary, I pointed this out because I wanted to exclude the fact that the keyboard set in Ubuntu was the standard US one. That would have explained why the tilde was working when you selected the US layout in the client.
Is the tilde the only problematic special character in your setup?
I'll try to make some tests with an Italian keyboard connecting from Windows to Ubuntu and I'll report my results.
No it's not necessary, I pointed this out because I wanted to exclude the fact that the keyboard set in Ubuntu was the standard US one. That would have explained why the tilde was working when you selected the US layout in the client.
Is the tilde the only problematic special character in your setup?
I'll try to make some tests with an Italian keyboard connecting from Windows to Ubuntu and I'll report my results.
It's not the only character. Also Pipe is not working. It is set to German on the Linux side. Still having it set to German on both sides, it doesn't work. When the client is set to US keyboard (my keyboard is German layout) it works correctly. Really strange.
The only reasoning I currently have is, that if the US keyboard layout is choosable/present there is some kind of fallback. If that's the case, it should be reproducible with an Italian keyboard.
Thanks for testing this behavior.
I can confirm there are issues with Italian keyboard mappings too on the Windows client. Regular Windows installer, version 4.3.1.
The right Alt key (necessary for typing "@" on an Italian keyboard for example) is mapped to Ctrl. In the following screenshot I'm hitting the right Alt key on an Italian keyboard:
This issue doesn't happen when I connect from the latest Flatpak client on Ubuntu.
I can add that everything works correctly if I boot Windows on the same server, so to recap:
Windows server - Windows client -> OK
Windows server - Ubuntu client -> OK
Ubuntu server - Ubuntu client -> OK
Ubuntu server - Windows client -> Keyboard mappings errors (Right Alt key on an Italian keyboard)
I just tried the latest nightly build of the Windows portable version (0.0.0.2267) and the issue persists.
Look what I found in the latest release of Sunshine for Windows (0.21.0)! Check the last option "Always Send Scancodes".
From the description it sounds exactly like what we're looking for, the problem is that there's no such option in the Linux version of Sunshine.
Ubuntu server - Windows client -> Keyboard mappings errors (Right Alt key on an Italian keyboard)
I can confirm this issue. Here’s the corresponding sunshine issue: https://github.com/LizardByte/Sunshine/issues/1224
Can confirm the issue with Sunshine Win10(cleint) > Win10(Server) Win10(client) > Arch(Server)
The described workarrounds above havent worked here. No matter what i set for Keyboard layouts i cant get a german Keyboard layout. For passwords and day to day use very difficult. Connection to same Host with a VNC client or a Teamviewer does work as ecxpected.
I can provide some more details. On my keyboard (configured as Swiss French on both Ubuntu host and Windows 10 client) the ~ is entered using the AltGr key. As I learned just now, this key does not exist on a US keyboard, which has a second Alt key. For me all the characters that use AltGr are broken. (Notably € | @ # ° § | ~ [ ] { } \ )
I had a look at Sunshine's debug log. Here's an annotated excerpt:
# left Ctrl
[2024:04:09:12:33:54]: Debug: --begin keyboard packet--
keyAction [00000003]
keyCode [80A2]
modifiers [02]
flags [00]
--end keyboard packet--
[2024:04:09:12:33:54]: Debug: --begin keyboard packet--
keyAction [00000004]
keyCode [80A2]
modifiers [00]
flags [00]
# (left) Alt
--end keyboard packet--
[2024:04:09:12:34:04]: Debug: --begin keyboard packet--
keyAction [00000003]
keyCode [80A4]
modifiers [04]
flags [00]
--end keyboard packet--
[2024:04:09:12:34:04]: Debug: --begin keyboard packet--
keyAction [00000004]
keyCode [80A4]
modifiers [00]
flags [00]
--end keyboard packet--
# AltGr (at the physical location where the US keyboard has a second Alt key)
[2024:04:09:12:34:11]: Debug: --begin keyboard packet--
keyAction [00000003]
keyCode [80A2]
modifiers [02]
flags [00]
--end keyboard packet--
[2024:04:09:12:34:11]: Debug: --begin keyboard packet--
keyAction [00000003]
keyCode [80A5]
modifiers [06]
flags [00]
--end keyboard packet--
[2024:04:09:12:34:11]: Debug: --begin keyboard packet--
keyAction [00000004]
keyCode [80A2]
modifiers [04]
flags [00]
--end keyboard packet--
[2024:04:09:12:34:11]: Debug: --begin keyboard packet--
keyAction [00000004]
keyCode [80A5]
modifiers [00]
flags [00]
--end keyboard packet--
# Ctrl
[2024:04:09:12:34:16]: Debug: --begin keyboard packet--
keyAction [00000003]
keyCode [80A3]
modifiers [02]
flags [00]
--end keyboard packet--
[2024:04:09:12:34:16]: Debug: --begin keyboard packet--
keyAction [00000004]
keyCode [80A3]
modifiers [00]
flags [00]
--end keyboard packet--
Moonlight (Windows) is sending a combination of A2 (left Ctrl) and A5 (right Alt). For this decoding, see here:
https://github.com/LizardByte/Sunshine/blob/7e26d2fd3064c6adf3e7a52a275046dd0cf194be/src/platform/linux/input.cpp#L289-L292
Apparently Windows interprets the combination of Ctrl+Alt as AltGr, which could explain why this issue specifically hits Linux servers. Also Gamestream is Windows-only. So this would even be "fine" protocol-wise in the original implementation.
What I find a bit puzzling is that in that list there is no specific entry for AltGr. (But I'm not an expert at keymaps, I just did some research.) Anyway, I think that for Linux hosts Moonlight could just send the specific AltGr code, if there is one, or Sunshine on Linux could re-interpret itself the combination of left Ctrl + right Alt as AltGR before emitting it as virtual input. (This would have the unfortunate side-effect of very slightly reducing the possible keymappings in games. AltGr and AltGr+Ctrl could no longer be differentiated?)
By googling I also found this:
// Windows does not have a specific key code for AltGr. We use the unused
// VK_OEM_AX to represent AltGr, matching the behaviour of Firefox on Linux.
VKEY_ALTGR = VK_OEM_AX,
https://chromium.googlesource.com/experimental/chromium/src/+/53.0.2785.12/ui/events/keycodes/keyboard_codes_win.h?autodive=0%2F%2F%2F%2F#186 Maybe it's useful in a future where compatibility with Gamestream is no longer a concern.
In fact the issue is not so much the distinction between Alt and AltGr, but rather the additional left Ctrl that moonlight is sending along.
Here is what xev is printing when I'm pressing and releasing AltGr on my physical keyboard connected to the host:
KeyRelease event, serial 37, synthetic NO, window 0x6800001,
root 0x55f, subw 0x0, time 30647870, (156,-22), root:(276,92),
state 0x84, keycode 37 (keysym 0xffe3, Control_L), same_screen YES,
XLookupString gives 0 bytes:
XFilterEvent returns: False
KeyRelease event, serial 37, synthetic NO, window 0x6800001,
root 0x55f, subw 0x0, time 30647870, (156,-22), root:(276,92),
state 0x80, keycode 108 (keysym 0xfe03, ISO_Level3_Shift), same_screen YES,
XKeysymToKeycode returns keycode: 92
XLookupString gives 0 bytes:
XFilterEvent returns: False
And this is when Moonlight sends AltGr:
KeyPress event, serial 37, synthetic NO, window 0x6800001,
root 0x55f, subw 0x0, time 30647814, (156,-22), root:(276,92),
state 0x0, keycode 37 (keysym 0xffe3, Control_L), same_screen YES,
XLookupString gives 0 bytes:
XmbLookupString gives 0 bytes:
XFilterEvent returns: False
KeyPress event, serial 37, synthetic NO, window 0x6800001,
root 0x55f, subw 0x0, time 30647814, (156,-22), root:(276,92),
state 0x4, keycode 108 (keysym 0xfe03, ISO_Level3_Shift), same_screen YES,
XKeysymToKeycode returns keycode: 92
XLookupString gives 0 bytes:
XmbLookupString gives 0 bytes:
XFilterEvent returns: False
KeyRelease event, serial 37, synthetic NO, window 0x6800001,
root 0x55f, subw 0x0, time 30647870, (156,-22), root:(276,92),
state 0x84, keycode 37 (keysym 0xffe3, Control_L), same_screen YES,
XLookupString gives 0 bytes:
XFilterEvent returns: False
KeyRelease event, serial 37, synthetic NO, window 0x6800001,
root 0x55f, subw 0x0, time 30647870, (156,-22), root:(276,92),
state 0x80, keycode 108 (keysym 0xfe03, ISO_Level3_Shift), same_screen YES,
XKeysymToKeycode returns keycode: 92
XLookupString gives 0 bytes:
XFilterEvent returns: False
(I did not test under Wayland.)
As you can see, AltGr is in fact handled correctly by Sunshine/Linux, but it's the spurious left Ctrl that breaks all the key combos.
Is it even necessary for Moonlight to try emulating AltGr as Alt+Ctrl? After all it's not just sending "some Alt key" but it does send an event for precisely RightAlt. Which should then be correctly interpreted by the host. Or is this some workaround for a limitation of nvidia's legacy Gamestream feature? Or maybe Moonlight's input layer is currently unable to collect such detailed input itself?
When I run Moonlight with SDL_EVENT_LOGGING=1 I find this output in Moonlight's log:
00:00:27 - SDL Info (0): SDL EVENT: SDL_KEYDOWN (timestamp=27536 windowid=2 state=pressed repeat=false scancode=224 keycode=1073742048 mod=64)
00:00:27 - SDL Info (0): SDL EVENT: SDL_KEYDOWN (timestamp=27537 windowid=2 state=pressed repeat=false scancode=230 keycode=1073742054 mod=576)
00:00:27 - SDL Info (0): SDL EVENT: SDL_KEYUP (timestamp=27589 windowid=2 state=released repeat=false scancode=224 keycode=1073742048 mod=512)
00:00:27 - SDL Info (0): SDL EVENT: SDL_KEYUP (timestamp=27590 windowid=2 state=released repeat=false scancode=230 keycode=1073742054 mod=0)
That's a single press of the AltGr key. So it seems to be an unfortunate limitation of its SDL input layer.
There's also an issue filed for SDL: https://github.com/libsdl-org/SDL/issues/7873
I'm just not sure what to make of it. I find the "resolution" quite unsatisfactory. Ctrl+Alt may be interpreted as AltGr by some OS, but that does not make it an equivalence.
E.g. in my main OS, Linux, AltGr+E gives me the € sign, whereas Ctrl+Alt+E could be assigned some random hotkey functionality. AltGr and Ctrl+Alt are indeed distinct. (I'm a bit surprised actually that Ctrl+Alt+E returns € in Windows.)
Also: isn't SDL supposed to be used for programming games and the like? Precise keyboard input is quite helpful in that domain, no? For example in many games I can bind actions specifically to the left Shift key etc.
So either someone should try to get this SDL issue reopened or Moonlight will have to implement a workaround. I.e. Detect LCtrl and RAlt in quick succession and just drop the LCtrl. If necessary Sunshine could then re-add LCtrl if it detects RAlt.
It turns out that SDL's keyboarding reading is platform-dependent. Here is what Moonlight gets from SDL when using Linux:
# AltGr
00:00:29 - SDL Info (0): SDL EVENT: SDL_KEYDOWN (timestamp=29961 windowid=2 state=pressed repeat=false scancode=230 keycode=1073742054 mod=512)
00:00:30 - SDL Info (0): SDL EVENT: SDL_KEYUP (timestamp=30036 windowid=2 state=released repeat=false scancode=230 keycode=1073742054 mod=0)
# AltGr+E -> €
00:00:39 - SDL Info (0): SDL EVENT: SDL_KEYDOWN (timestamp=39060 windowid=2 state=pressed repeat=false scancode=230 keycode=1073742054 mod=512)
00:00:39 - SDL Info (0): SDL EVENT: SDL_KEYDOWN (timestamp=39521 windowid=2 state=pressed repeat=false scancode=8 keycode=101 mod=512)
00:00:39 - SDL Info (0): SDL EVENT: SDL_KEYUP (timestamp=39598 windowid=2 state=released repeat=false scancode=8 keycode=101 mod=512)
00:00:40 - SDL Info (0): SDL EVENT: SDL_KEYUP (timestamp=40138 windowid=2 state=released repeat=false scancode=230 keycode=1073742054 mod=0)
The corresponding Sunshine (also Linux) log excerpt is as follows:
# Single AltGr key
[2024:04:09:19:54:12]: Debug: --begin keyboard packet--
keyAction [00000003]
keyCode [80A5]
modifiers [04]
flags [00]
--end keyboard packet--
[2024:04:09:19:54:12]: Debug: --begin keyboard packet--
keyAction [00000004]
keyCode [80A5]
modifiers [00]
flags [00]
--end keyboard packet--
# AltGr+E -> €
[2024:04:09:19:54:21]: Debug: --begin keyboard packet--
keyAction [00000003]
keyCode [80A5]
modifiers [04]
flags [00]
--end keyboard packet--
[2024:04:09:19:54:22]: Debug: --begin keyboard packet--
keyAction [00000003]
keyCode [8045]
modifiers [04]
flags [00]
--end keyboard packet--
[2024:04:09:19:54:22]: Debug: --begin keyboard packet--
keyAction [00000004]
keyCode [8045]
modifiers [04]
flags [00]
--end keyboard packet--
[2024:04:09:19:54:22]: Debug: --begin keyboard packet--
keyAction [00000004]
keyCode [80A5]
modifiers [00]
flags [00]
--end keyboard packet--
In this case everything is fine: a single keypress (AltGr) corresponds to a single scancode, which is in turn converted to a keycode and everyone's happy.
Out of curiosity I also tested Moonlight(Linux) connecting to Sunshine(Windows). Moonlight is still only sending RightAlt to Sunshine and it's working fine! Here is Sunshine's log:
[2024:04:09:20:09:07]: Debug: --begin keyboard packet--
keyAction [00000003]
keyCode [80A5]
modifiers [04]
flags [00]
--end keyboard packet--
[2024:04:09:20:09:07]: Debug: --begin keyboard packet--
keyAction [00000004]
keyCode [80A5]
modifiers [00]
flags [00]
--end keyboard packet--
[2024:04:09:20:09:14]: Debug: --begin keyboard packet--
keyAction [00000003]
keyCode [80A5]
modifiers [04]
flags [00]
--end keyboard packet--
[2024:04:09:20:09:14]: Debug: --begin keyboard packet--
keyAction [00000003]
keyCode [8045]
modifiers [04]
flags [00]
--end keyboard packet--
[2024:04:09:20:09:15]: Debug: --begin keyboard packet--
keyAction [00000004]
keyCode [8045]
modifiers [04]
flags [00]
--end keyboard packet--
[2024:04:09:20:09:15]: Debug: --begin keyboard packet--
keyAction [00000004]
keyCode [80A5]
modifiers [00]
flags [00]
--end keyboard packet--
This seems to work fine with one exception which still is misbehaving. Ironically it's ~ which happens to be the original report... At least on the Swiss French layout, ~ is actually a "dead key". In order to get a plain ~ I have to input AltGr + ^ and then a space (or just press the ^ key again). For an ñ, I'll type AltGr + ^, n. Now with the broken behaviour I have to press and hold AltGr, then hit ^ twice instead of once and then only let go of both keys and then only press n.
Apart from that, it seems that the spurious LeftControl that SDL(Windows) emits could simply be suppressed. Both Linux and Windows seem to be fine if an AltGr results in just a RightAlt. (I'm not sure about the weird ~ dead key combo)
In one of my previous comments I linked to a closed SDL issue. I did miss this one which is still open and properly acknowledges it: https://github.com/libsdl-org/SDL/issues/5685 It's also impacting other projects.
I'd also like to add that the original report is about entering ~ using a German keyboard layout. This particular layout has ~ on AltGr + +. So this whole issue is all about the AltGr key not being recognized correctly on Windows.
Please try the latest nightly which includes an upstreamed fix for AltGr handling in SDL on Windows.
https://ci.appveyor.com/project/cgutman/moonlight-qt/builds/50531781/job/ia9h6iixmq395pbk/artifacts
