Modifiers in modifier functions get dropped in certain programs/desktop environments

[caksoylar]

Edited to make it more generic: This issue is concerning the timing of keys sent by ZMK when modifier functions are involved, such as &kp LA(TAB), &kp HASH (equal to &kp LS(N3)) etc. Some applications/desktop environments don't seem to like how ZMK sends these keys and sometimes ignores the modifier parts of these keycodes, leading to unexpected behavior. So far issues have been observed in:

• Windows classic RDP app (see original post below)
• Gnome under Wayland (upstream issue linked below)

Original post

There are issues using sending key presses with modifier functions and overall issues with held modifiers when using the RDP app ("Remote Desktop Connection") on Windows 10. I use it for connecting to a local VM with Hyper-V using vmconnect.exe, so there is no network latency. For instance when sending a key press like LALT(TAB) the modifier is almost always ignored. When using regular modifiers like Shift, they are also not detected some of the time.

Note that there is also a different RDP app ("Remote Desktop Classic" on MS Store) that doesn't seem to have the issue, so it is likely related to the peculiar handling of keyboard inputs of the former app. However this app doesn't allow connections to Hyper-V VMs.

Here is me sending &kp UNDER in the host (not using RDP) on https://config.qmk.fm/#/test:

Here it is on the remote VM through the classic RDP app:

Worth noting that the minus keycode is also reported differently, but I don't know if that has any relevance.

Could it potentially be helped by a parameter like tap-duration-ms as proposed in https://github.com/zmkfirmware/zmk/issues/756? Or maybe the RDP app expects keycodes to be spaced apart in time?

[okke-formsma]

Could you show the timings when you press a keycode like that in QMK? The solution for #756 does not apply here, as that is for hold-taps and does not apply to shifted keycodes.

[caksoylar]

This is a bit interesting, here is QMK underscore keycode on local (no RDP), one short tap and one long tap:

This is the same on remote:

First of all the order of key ups are different, QMK seems to release shift ~16ms after the minus, but ZMK releases shift before the minus (they seem to be held for the exact same duration).

I also have issues getting manual shift holds recognized with ZMK if I am not pressing it very deliberately which doesn't happen with QMK. Let me know if there is anything I can share to help debug that.

[okke-formsma]

Are you using ZMK while connected through bluetooth or usb? Could you verify the issue is the same using both connection methods?

It seems QMK is not processing the shift and '-' keyup events quickly, but adding some 16ms delays in between. To emulate this we may have to somehow queue keypresses instead of processing them as fast as possible.

I've never had issues with 'deliberate pressing shift' or not, so can't really say anything about that.

[caksoylar]

Are you using ZMK while connected through bluetooth or usb? Could you verify the issue is the same using both connection methods?

I primarily use BT but I can confirm that the behavior is the same on USB.

I've never had issues with 'deliberate pressing shift' or not, so can't really say anything about that.

To clarify this is only over RDP; I don't have any such issues outside the RDP app.

I tried the short tap and long tap for &kp UNDER again over USB, on the remote VM. The key-up event for Shift ~5ms after key-down is very consistent:

I also noticed that if I hold the &kp UNDER button down long enough (>500ms), it detects an additional shift down event as in the long tap event above (red highlighted part). This doesn't happen outside RDP or with QMK inside or outside RDP.

I guess the RDP client is doing its own held key detection with some thresholds and looking at event ordering, and ZMK's order of events somehow doesn't play well with it.

[halfurness]

I'm seeing this behaviour too, LC() isn't working for me in RDP

[caksoylar]

Potentially related: https://github.com/qmk/qmk_firmware/issues/13708

[abstracterror]

A workaround that mostly seems to work for individual &kp bindings is to define a macro for each that presses the modifier key, taps the unmodified key code, and then releases the modifier key.

I set wait-ms to 16ms in the macros, based on the numbers given in the comments above. I still get occasional unshifted characters, but much less frequently.

Unfortunately the issue also applies to &caps_word, with some of the letters ending up shifted, and some not.

[rambip]

I have a similar timing issue on gnome (wayland + fedora) I defined a "tap-modifier" to output colon, but in gnome it outputs semicolon

But even stranger, in sway (a wayland tiling window manager) I never get tbat error.

I get it with bluetooth and usb.

Are there values I can tweak in the config for timing ?

[catdevnull]

@rambip I have a similar problem with sticky keys; in Sway they are reliable, but in GNOME they are hit or miss.

[catdevnull]

Sorry for the shitty video, but here's a demo:

muestra2.webm

[catdevnull]

Even in GNOME, it doesn't happen under Qt apps. This also affects things like &kp AT, yielding 2 instead.

[hallettj]

Because I'm having the same problem in Gnome & Wayland I used @abstracterror's helpful macro workaround. I agree it makes modified key presses work much more consistently, but it is still not perfect. I didn't want to write a macro for every shifted key I use, so I wrote this C preprocessor macro that generates ZMK macros:

#define SHIFTED(sym, macro_label) s_##sym: s_##sym {\
        label = macro_label;\
        compatible = "zmk,behavior-macro";\
        wait-ms = <50>;\
        #binding-cells = <0>;\
        bindings\
            = <&macro_press &kp LSHFT>\
            , <&macro_press &kp sym>\
            , <&macro_release &kp LSHFT>\
            , <&macro_pause_for_release>\
            , <&macro_release &kp sym>\
            ;\
    }

My macros section looks like this:

    macros {
        SHIFTED(EXCL, "shifted_excl");
        SHIFTED(AT, "shifted_at");
        SHIFTED(HASH, "shifted_hash");
        /* etc. */
    };

That lets me use bindings of the form &s_EXCL, &s_AT, &s_HASH, etc.

I'm passing in the shifted codes (like EXCL instead of N1) which is technically redundant. That helps me to keep my configuration readable, and it doesn't seem to hurt anything.

I tried to derive the label from the first macro argument, but I couldn't figure out how to get macro-level string interpolation or concatenation to work. So I gave up, and used a second argument for the label.

I hope this is helpful for someone in a similar situation.

Edit 2022-12-15: I edited the preprocessor macro above to release LSHFT before the pause_for_release step to prevent the shift modifier from bleeding into follow-up key presses. I also increased the wait time from 35 to 50 ms because I'm still seeing occasional unshifted characters.

[catdevnull]

As far as I can tell, this is an upstream issue: https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/5890

[pergamomo]

Until this is fixed in Gnome is there a way to adjust the internal timing globally so we don't need to write a macro for every shifted key?

[catdevnull]

Annoyingly, it doesn't seem to affect QMK :(

[hallettj]

As far as I can tell, this is an upstream issue: https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/5890

I've just noticed a comment in that issue that suggests running pkill ibus as a workaround to the problem in Gnome. That seems to work for me!

It looks like ibus provides features like input methods for alphabets other than the Latin one, and for emoji. I'm not sure, but it might also be required for switching input methods. I don't know much about the consequences of disabling it. I also don't see any built-in option to disable it permanently. Comments I'm seeing in searches say that "Gnome requires ibus".

[pergamomo]

@hallettj Sadly this didn't work for me. Maybe because I'm using English (intl. with AltGr dead keys)?

Killing iBus also killed the keyboard input so I had to reboot.

[catdevnull]

@pergamomo I'm the original poster of that workaround, and I'm using the same layout. Not sure what ibus really does, but I read something about input methods and thought that this could be the problem.

[JMHAVANCE]

Not sure if this helps anyone, but using Remote Desktop on Windows (in windowed mode) solved this issue for me connecting to both other Windows machines as well as Ubuntu machines.

Full screen: issues with hold-tap and layers. Windowed: normal

(even when stretching the window across my entire screen)

[caksoylar]

Also relevant: https://github.com/qmk/qmk_firmware/pull/19449

From the PR description it sounds like sending modifier events separate from other key events helps with both RDP and Gnome scenarios.

[abliss]

@JMHAVANCE I have a theory about why it works for you in windowed mode but not fullscreen: There's an option in the windows RDP settings called something like "Apply windows hotkeys locally (e.g. alt-tab)". (In the config file, I think this is "keyboardhook"). It has three values: "always" (1), "never" (0), and "only when full-screen" (??). Maybe you have "only when full-screen" selected and thus windowed/fullscreen makes the difference for you? For me, I found that setting this value to "never" fixes the problem even when I am fullscreen (which I basically have to be, because the weird layout of my monitors makes it impossible for a window to cover all/only my active pixels.)

[JMHAVANCE]

@abliss thanks! this did the trick for me as well!

[caksoylar]

Looks like the GNOME issue might be resolved in the near future, the issue was closed by: https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/3044. Not sure when this will make its way to a release.

[catdevnull]

Seems to be in Mutter 45 (I believe GNOME 45 is still beta)

[kevinpastor]

Is there a way to modify the timing as QMK does with TAP_CODE_DELAY? This is what they suggest to modify for problems like this: https://docs.qmk.fm/#/mod_tap?id=caveats.

[manna-harbour]

Is there a way to modify the timing as QMK does with TAP_CODE_DELAY?

https://github.com/zmkfirmware/zmk/issues/1443

https://github.com/zmkfirmware/zmk/issues/1444#issuecomment-1291525634

[kevinpastor]

@manna-harbour, I've looked what MIRYOKU_KLUDGE_TAPDELAY enables in Miryoku but can't grasp what it actually does. From my understanding, you're defining a macro that simply presses and releases a given key. Are you overwritting the default constants ZMK is providing for letters and some symbols (e.g. A or SQT)? I can't find where the wait time is defined in your repo; does this flag modifies the default wait time of a macro?

Any information would greatly help. Thanks in advance!

[kevinpastor]

I'm sorry, I didn't realize that the link I sent was about Mod-Tap specifically. Would there be something similar for modifier functions?

[kevinpastor]

I've tried a lot of things and finally realized that the workaround of a macro slowing things down did work over USB but not Bluetooth.

I am now wondering if one of the system configuration for Bluetooth could help here. I've tried to understand what CONFIG_ZMK_BLE_CONSUMER_REPORT_QUEUE_SIZE, CONFIG_ZMK_BLE_KEYBOARD_REPORT_QUEUE_SIZE, CONFIG_ZMK_BLE_THREAD_PRIORITY, and CONFIG_ZMK_BLE_THREAD_STACK_SIZE do by reading ZMK and Zephyr's documentation, but I'm totally lost. Could these settings help in the timing of keypresses? Setting CONFIG_ZMK_BLE_CONSUMER_REPORT_QUEUE_SIZE to 1 seemed to have helped a bit, but I'm not sure if it's unrelated.

For anyone wondering what my macro looks like, here's the snippet I use which makes use of zmk-nodefree-config helpers:

ZMK_BEHAVIOR(ls, macro_one_param, \
    wait-ms = <0>; \
    tap-ms = <0>; \
    bindings = \
        <&macro_press &kp LSHFT>, \
        <&macro_wait_time 100>, \
        <&macro_param_1to1>, \
        <&macro_tap &kp MACRO_PLACEHOLDER>, \
        <&macro_wait_time 100>, \
        <&macro_release &kp LSHFT>; \
)

// Using the binding in the keymap
&ls N7

[caksoylar]

Note that &macro_wait_time 100 doesn't mean "wait here for 100ms", it means "change the wait time between actions to 100ms after this". So I am guessing the timing between shift press and param tap isn't working like you'd want it to. Looks like you can simply use wait-ms = <100>; and get rid of the macro_wait_time bindings. I'd also set tap-ms to a larger value like 10 if it still isn't working.