(Porting/compatibility issue) for virtual keyboards that don't send key press events for modifiers, SDL2 should insert them

[ell1e]

For virtual keyboards that don't send key press events for modifiers (which seems to always be the case for Linux/Wayland touch devices, at least it is with Phosh+squeekboard on the PinePhone), SDL2 should insert them if it detects them being missing. Example: if SDL2 prepares to send an SDLK_a key down with CTRL modifier enabled but didn't previously send an SDLK_LCTRL or SDLK_RCTRL event, it should insert a fake one right before that SDLK_a event. Edit: as for release, that would probably best done right after each keypress again, since multi letter key shortcuts can't usually be entered on these touch keyboards anyway.

This seems to be impossible to solve correctly at the app level due to the lacking information about which keyboard the key presses come from, I'm pretty sure with multiple keyboards this would then lead to an incorrect state in corner cases. Also, solving it at the app level kind of goes against what I'm hoping this would achieve:

I'm just hoping naive, simple SDL2 applications that have simple UI, happen to work on a phone screen, and may naively use shortcuts based on sdl key down/up tracking instead of checking the modifier flag still "just work". I ran into an application of my own running into this very same bug, so I think it's an easy detail to miss. Devs just don't really test for some types of keys on only some platforms not emitting up/down events.

Edit: I'm aware this doesn't fix anything for cases where CTRL needs to be pressed on its own, like for crouching controls or alike. But such games aren't usually playable with a touch keyboard in practice. However, something like manager games or media player apps might have their keyboard shortcuts saved by this

[flibitijibibo]

This is actually possible on Wayland, but it would require the assumption that this is touch only - the compositor will give us a text input manager and not a keyboard handle, and in this case we could possibly reuse the text information to send presses, but I would not want to be the person in charge of making that dictionary...

[ell1e]

Wouldn't it make more sense to do it generically? Track for all keyboards whether any CTRL down (for either lctrl or rctrl) was ever passed through, and if not but a ctrl modifier is set on a key from that keyboard, prefix the event with an LCTRL keydown and sent an LCTRL key release right after (or in the next frame/with a tiny delay maybe).

I imagine this could also help in other scenarios where for some reason the key presses and modifier state become desynced, that doesn't seem like the worst thing to try to prevent more generally. Or is it?

[Kontrabant]

Modifiers are handled in #6431 now. The code that was recently removed in #6403 was actually perfect for handling virtual keyboard modifiers that toggle modifier state, but don't send key presses.

[slouken]

Out of curiosity, what virtual keyboards send control keys but don't send the control key separately?

[ell1e]

Phosh's squeekboard which has a "Terminal" layout with CTRL and ALT modifier buttons, it's probably one of the most common one encountered on Linux phones. There's some info on this ticket here although I don't fully understand why it doesn't just send these key presses other than it's likely not being changed.

[Kontrabant]

Beyond that, keyboards can also enable things such as shiftlock (distinct from capslock), which causes everything to act as though shift is pressed even though the shift keys are not actually down, and depending on how it's configured, pressing the shift key in this state will turn the modifier off, so the actual shift modifier state in inverse to the state of the shift keys. In the case of Wayland, any modifier can be latched/locked and thus active without the associated key being pressed.

In general, key modifiers being active doesn't necessarily reflect the actual state of the modifier keys anymore.

[ell1e]

I see, that capslock thing does sound tricky, although afaik squeekboard currently doesn't behave like that. I don't know, I just think one side (SDL2 or Wayland keyboards/squeekboard) should try to address this somehow. Just because it's such an unnecessary porting pitfall.

[dos1]

I don't fully understand why it doesn't just send these key presses

Because for many keys, it doesn't send key presses at all. It generates text input events and sends "a" or "A" or even "🤩️" directly without pressing any buttons. It has limited support for key presses to handle things like CTRL key or arrow buttons, but there's no SHIFT key there at all (there's only a button that looks like it has similar function, but it's actual purpose is to switch key layouts which in fact don't have to match each other at all like on a real keyboard, so it doesn't really map to real SHIFT).