JigsawWM stability of low-level keyboard interface

When pressing sequences of keys, it would be expected that the keyboard interface behaves exactly as the native Windows keyboard interface does. This means that slow and fast typists would recognize no change in the behavior of the keyboard interface when working inside a specific application. Even on multiple sequential key presses with overlap of keys, the interface should behave just like native windows. That would be the optimal case, I guess..

With JigsawWM I see some issues:

one example is that when e.g. changing to neovim application and trying to issue commands in command mode, the command mode is suddenly closed when trying to insert a standard Windows path containing a \ BACHSLASH character.
another example is, when typing fast and thus sometimes holding two keys down simultaneously while typing a word. The keys don't get recognized in a reliably stable way. So, sometimes the <SPACE> character is missing. Or other keys.

Aug 12 '23 08:08 nnako

@nnako Thanks for reporting the issue. I would need your key configuration related to the problematic behavior to analyze what went wrong,

Aug 14 '23 04:08 klesh

thanks. please see #5 for the description of my configuration.

Aug 14 '23 10:08 nnako

Hi @klesh ,

here you see a video (please excuse my spoken English, hope you understand anyway) showing a severe keyboard issue which seems to occur connected to this situations:

pressing a couple of keys quickly while they might overlap at times this seems to trigger unpredictable behaviours within the active application
especially pressing backslash key leads to unexpected behaviour and even seems to neutralize / freeze the <WIN> key for further actions.

the halt / neutralization of the <WIN> key seems to get healed when randomly pressing <CTRL>, <ALT> and <WIN> keys. Maybe even simultaneously in one or the other combination. Unfortunately, even switching on DEBUG mode on PowerShell does just log the ordinary actions (switching of application windows).

here the video:

https://github.com/klesh/JigsawWM/assets/5845602/589d1275-04c1-4e64-b453-37ced2acbd3a

During the video you saw other windows pop up. These normally get opened when actively pressing the <WIN> key. But in my case I didn't press that key. Just randomly typed some "ordinary" letters like <A>, <S>, <D>, <F>, <Y>, <X>, <Q> and <W> . Partially simultaneously and with different typing speeds.

Aug 16 '23 09:08 nnako

Thanks for the video, it is quite informative, I can reproduce the problem on my local machine. Will see to it.❤️

Aug 17 '23 13:08 klesh

I just updated the main branch, the problem should be fixed now, feel free to reopen the issue if is wasn't.

Aug 17 '23 13:08 klesh

Hi @klesh , thanks for your efforts. Unfortunately, you solved #5 . So, now with the current main branch, the "bleeding" has stopped. I would like to re-open this issue. The essence of the issue was:

a kind of unpredictable reaction when pressing "normal" keys like e.g. <A>, <S>, <D>, <F>, <Y>, <X>, <Q> and <W> in an partly overlapping manner. Now, when pressing and holding <Y> and then additionally pressing <X> , in Neovim the letters <F2> appear. As if the key <F12> had been pressed. pressing <Y> and then <C>in the same manner lets <F3> appear. In Germany, we use <Y> in the lower row left of <X>. Dunno, if this might cause a difference.
pressing \ causes the <WIN> key to loose its function until the keys <CTRL>, <WIN> and <ALT> had been pressed multiple times (even overlapping). After that, the <WIN> key works properly again. But the <\> key does not show any physical representation on the screen.

These issues are still visible on my computer. Sorry for describing too much within the video above. Please have a look from 1:45 . There you see the issue. It is still active.

Aug 17 '23 21:08 nnako

Hi, @nnako

Hi @klesh , thanks for your efforts. Unfortunately, you solved #5 . So, now with the current main branch, the "bleeding" has stopped. I would like to re-open this issue. The essence of the issue was:

a kind of unpredictable reaction when pressing "normal" keys like e.g. <A>, <S>, <D>, <F>, <Y>, <X>, <Q> and <W> in an partly overlapping manner. Now, when pressing and holding <Y> and then additionally pressing <X> , in Neovim the letters <F2> appear. As if the key <F12> had been pressed. pressing <Y> and then <C>in the same manner lets <F3> appear. In Germany, we use <Y> in the lower row left of <X>. Dunno, if this might cause a difference.

Hi, It is the default configuration, then when the <Y> key is held and tap the <X> key will produce a <F2>

https://github.com/klesh/JigsawWM/blob/main/examples/jmk.pyw#L28 switch to layer 3 when held
https://github.com/klesh/JigsawWM/blob/main/examples/jmk.pyw#L102 map <X> to <F2> key

You can change the configuration to fit your need, all files in the examples folder are meant for demonstration, please copy them somewhere and change it as you see fit.

As for the unpredictable behavior when fast typing, it is a little bit complicated to explain, it is unavoidable when turning an Orindary Key to react differently for Tapping and Holding. You may take a look at this article https://docs.qmk.fm/#/tap_hold

pressing \ causes the <WIN> key to loose its function until the keys <CTRL>, <WIN> and <ALT> had been pressed multiple times (even overlapping). After that, the <WIN> key works properly again. But the <\> key does not show any physical representation on the screen.

These issues are still visible on my computer. Sorry for describing too much within the video above. Please have a look from 1:45 . There you see the issue. It is still active.

As of \ key... I don't know what was wrong. can you try running command py -m jigsawwm.w32.hook and press it and share the console output here? Thanks.

Aug 18 '23 02:08 klesh

when entering that phrase, I get loads of output. So fast that I can not even press <CTRL> + <C> fast enough to be able to not fill my console buffer entirely. During execution of py -m jigsawwm.w32.hook mouse movements are very slow (low frequency). Here is a tiny part of what I see:

==================================
==================================
[01:09.949618] UNKNOWN                          590960 ido:      0 idc:     -1 Eingabeaufforderung - py  -m jigsawwm.w32.hook

hwnd         : 590960
title        : Eingabeaufforderung - py  -m jigsawwm.w32.hook
pid          : c_ulong(4404)
class name   : ConsoleWindowClass
exe path     : C:\Windows\System32\cmd.exe
style        : BORDER, CLIPSIBLINGS, DLGFRAME, GROUP, MAXIMIZEBOX, SIZEBOX, SYSMENU, VISIBLE, VSCROLL
exstyle      : ACCEPTFILES, APPWINDOW, LAYERED, WINDOWEDGE
is_cloaked   : False
rect         : -2630 1278 -946 1589
bound        : -2623 1278 -953 1582
is_app_window: True
is_manageable: True
is_evelated  : False
==================================
==================================
[01:09.952599] UNKNOWN                          590960 ido: 131006464 idc: 131006496 Eingabeaufforderung - py  -m jigsawwm.w32.hook

hwnd         : 590960
title        : Eingabeaufforderung - py  -m jigsawwm.w32.hook
pid          : c_ulong(4404)
class name   : ConsoleWindowClass
exe path     : C:\Windows\System32\cmd.exe
style        : BORDER, CLIPSIBLINGS, DLGFRAME, GROUP, MAXIMIZEBOX, SIZEBOX, SYSMENU, VISIBLE, VSCROLL
exstyle      : ACCEPTFILES, APPWINDOW, LAYERED, WINDOWEDGE
is_cloaked   : False
rect         : -2630 1278 -946 1589
bound        : -2623 1278 -953 1582
is_app_window: True
is_manageable: True
is_evelated  : False
==================================
==================================
[01:09.956676] UNKNOWN                          590960 ido:      0 idc:     -1 Eingabeaufforderung - py  -m jigsawwm.w32.hook

hwnd         : 590960
title        : Eingabeaufforderung - py  -m jigsawwm.w32.hook
pid          : c_ulong(4404)
class name   : ConsoleWindowClass
exe path     : C:\Windows\System32\cmd.exe
style        : BORDER, CLIPSIBLINGS, DLGFRAME, GROUP, MAXIMIZEBOX, SIZEBOX, SYSMENU, VISIBLE, VSCROLL
exstyle      : ACCEPTFILES, APPWINDOW, LAYERED, WINDOWEDGE
is_cloaked   : False
rect         : -2630 1278 -946 1589
bound        : -2623 1278 -953 1582
is_app_window: True
is_manageable: True
is_evelated  : False
==================================
==================================
[01:09.958676] UNKNOWN                          590960 ido: 131006464 idc: 131006505 Eingabeaufforderung - py  -m jigsawwm.w32.hook

hwnd         : 590960
title        : Eingabeaufforderung - py  -m jigsawwm.w32.hook
pid          : c_ulong(4404)
class name   : ConsoleWindowClass
exe path     : C:\Windows\System32\cmd.exe
style        : BORDER, CLIPSIBLINGS, DLGFRAME, GROUP, MAXIMIZEBOX, SIZEBOX, SYSMENU, VISIBLE, VSCROLL
exstyle      : ACCEPTFILES, APPWINDOW, LAYERED, WINDOWEDGE
is_cloaked   : False
rect         : -2630 1278 -946 1589
bound        : -2623 1278 -953 1582
is_app_window: True
is_manageable: True
is_evelated  : False
==================================
==================================
[01:09.961675] UNKNOWN                          590960 ido:      0 idc:     -1 Eingabeaufforderung - py  -m jigsawwm.w32.hook

hwnd         : 590960
title        : Eingabeaufforderung - py  -m jigsawwm.w32.hook
pid          : c_ulong(4404)
class name   : ConsoleWindowClass
exe path     : C:\Windows\System32\cmd.exe
style        : BORDER, CLIPSIBLINGS, DLGFRAME, GROUP, MAXIMIZEBOX, SIZEBOX, SYSMENU, VISIBLE, VSCROLL
exstyle      : ACCEPTFILES, APPWINDOW, LAYERED, WINDOWEDGE
is_cloaked   : False
rect         : -2630 1278 -946 1589
bound        : -2623 1278 -953 1582
is_app_window: True
is_manageable: True
is_evelated  : False
==================================
==================================
[01:09.964676] UNKNOWN                          590960 ido: 131006464 idc: 131006563 Eingabeaufforderung - py  -m jigsawwm.w32.hook

hwnd         : 590960
title        : Eingabeaufforderung - py  -m jigsawwm.w32.hook
pid          : c_ulong(4404)
class name   : ConsoleWindowClass
exe path     : C:\Windows\System32\cmd.exe
style        : BORDER, CLIPSIBLINGS, DLGFRAME, GROUP, MAXIMIZEBOX, SIZEBOX, SYSMENU, VISIBLE, VSCROLL
exstyle      : ACCEPTFILES, APPWINDOW, LAYERED, WINDOWEDGE
is_cloaked   : False
rect         : -2630 1278 -946 1589
bound        : -2623 1278 -953 1582
is_app_window: True
is_manageable: True
is_evelated  : False
==================================
==================================
[01:09.967675] UNKNOWN                          590960 ido: 131006564 idc: 458828 Eingabeaufforderung - py  -m jigsawwm.w32.hook

hwnd         : 590960
title        : Eingabeaufforderung - py  -m jigsawwm.w32.hook
pid          : c_ulong(4404)
class name   : ConsoleWindowClass
exe path     : C:\Windows\System32\cmd.exe
style        : BORDER, CLIPSIBLINGS, DLGFRAME, GROUP, MAXIMIZEBOX, SIZEBOX, SYSMENU, VISIBLE, VSCROLL
exstyle      : ACCEPTFILES, APPWINDOW, LAYERED, WINDOWEDGE
is_cloaked   : False
rect         : -2630 1278 -946 1589
bound        : -2623 1278 -953 1582
is_app_window: True
is_manageable: True
is_evelated  : False
==================================
==================================
[01:09.969676] UNKNOWN                          590960 ido:      0 idc:     -1 Eingabeaufforderung - py  -m jigsawwm.w32.hook

hwnd         : 590960
title        : Eingabeaufforderung - py  -m jigsawwm.w32.hook
pid          : c_ulong(4404)
class name   : ConsoleWindowClass
^C

Aug 18 '23 13:08 nnako

Hi, It is the default configuration, then when the <Y> key is held and tap the <X> key will produce a <F2>

https://github.com/klesh/JigsawWM/blob/main/examples/jmk.pyw#L28 switch to layer 3 when held

https://github.com/klesh/JigsawWM/blob/main/examples/jmk.pyw#L102 map <X> to <F2> key

You can change the configuration to fit your need, all files in the examples folder are meant for demonstration, please copy them somewhere and change it as you see fit.

Hmmm... please, could you explain how it comes that I get the defaults of jmk.pyw activated when I started wm.pyw? How do those configurations relate to each other? And how can I take, for example, wm.pyw as a basis for my own configurations and put it into my own location, then, without still activating the defaults within jmk.pyw?

Aug 18 '23 14:08 nnako

As for the unpredictable behavior when fast typing, it is a little bit complicated to explain, it is unavoidable when turning an Orindary Key to react differently for Tapping and Holding. You may take a look at this article https://docs.qmk.fm/#/tap_hold

you sent the link for explanation of technique "holding and tapping", right? generally, there is no connection between JigsawWM and the QMK firmware and drivers, right? As far as I see it, the concept for "holding and tapping" is now understood. Not too complicated ;-) .

Aug 18 '23 21:08 nnako

For the py -m jigsawwm.w32.hook, I failed to mention that you may need to comment Line 352 to 363 so that only the mouse and keyboard events would be printed.
The reason why wm.pyw causes the jmk.pyw get activated with its configuration is that wm.pyw imports jmk.pyw because it relies on the hks variable for registering hotkeys and jmk.pyw would register itself and gets activated automatically. The .pyw should act more like a configuration rather than a feature, so it would be better to place your new feature under the jmk folder and configure it with a .pyw file in the examples folder. So, you should avoid importing the jmk.pyw but initialize the jmk in your own .pyw file if it is undesirable.
There is no physical connection between the JigsawWM and the QMK for sure. I just try to mimic some basic features from the great QMK.😊

Aug 19 '23 12:08 klesh

Hi @klesh ,

I have checked as you asked me to. Now, I try to explain what I did in order to see the keyboard low-level debug output, after I started wm.pyw from another terminal until I ended it using <CTRL> + <C>:

change to Neovim application...

LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6459015 extra: 554696706
TAB             WM_SYSKEYDOWN  : vkCode   9 scanCode   0 flags:  48, time: 6459171 extra: 554696706
TAB             WM_SYSKEYUP    : vkCode   9 scanCode   0 flags: 176, time: 6459250 extra: 554696706
TAB             WM_SYSKEYDOWN  : vkCode   9 scanCode   0 flags:  48, time: 6459625 extra: 554696706
TAB             WM_SYSKEYUP    : vkCode   9 scanCode   0 flags: 176, time: 6459734 extra: 554696706
TAB             WM_SYSKEYDOWN  : vkCode   9 scanCode   0 flags:  48, time: 6460015 extra: 554696706
TAB             WM_SYSKEYUP    : vkCode   9 scanCode   0 flags: 176, time: 6460125 extra: 554696706
LMENU           WM_KEYUP       : vkCode  a4 scanCode   0 flags: 144, time: 6460250 extra: 554696706

press some keys <:> (colon using <SHIFT> + <.> on a German keyboard), then <A>, <S>, ... which results in...

RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462000 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462500 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462546 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462562 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462593 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462640 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462687 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462703 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462750 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462781 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462796 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462828 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462859 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462906 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462937 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6462968 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463000 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463031 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463062 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463093 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463125 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463156 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463187 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463234 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463265 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463296 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463328 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463375 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463390 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463421 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463453 extra: 554696706
RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode   0 flags:  17, time: 6463484 extra: 554696706
OEM_PERIOD      WM_KEYDOWN     : vkCode  be scanCode   0 flags:  16, time: 6463531 extra: 554696706
OEM_PERIOD      WM_KEYUP       : vkCode  be scanCode   0 flags: 144, time: 6463734 extra: 554696706
RSHIFT          WM_KEYUP       : vkCode  a1 scanCode   0 flags: 145, time: 6464046 extra: 554696706
A               WM_KEYDOWN     : vkCode  41 scanCode   0 flags:  16, time: 6465359 extra: 554696706
A               WM_KEYUP       : vkCode  41 scanCode   0 flags: 144, time: 6465359 extra: 554696706
S               WM_KEYDOWN     : vkCode  53 scanCode   0 flags:  16, time: 6465953 extra: 554696706
S               WM_KEYUP       : vkCode  53 scanCode   0 flags: 144, time: 6465953 extra: 554696706
D               WM_KEYDOWN     : vkCode  44 scanCode   0 flags:  16, time: 6466546 extra: 554696706
D               WM_KEYUP       : vkCode  44 scanCode   0 flags: 144, time: 6466562 extra: 554696706
F               WM_KEYDOWN     : vkCode  46 scanCode   0 flags:  16, time: 6467140 extra: 554696706
F               WM_KEYUP       : vkCode  46 scanCode   0 flags: 144, time: 6467140 extra: 554696706

now, I press BACKSLASH by pressing <R-ALT> + <SZLIG> (the latter one being the appropriate key on German keyboards. pressing it with <SHIFT> would result in a question mark). which results in ...

LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6468703 extra: 554696706
RMENU           WM_KEYDOWN     : vkCode  a5 scanCode   0 flags:  16, time: 6468703 extra: 554696706
RMENU           WM_KEYDOWN     : vkCode  a5 scanCode   0 flags:  16, time: 6469218 extra: 554696706
RMENU           WM_KEYDOWN     : vkCode  a5 scanCode   0 flags:  16, time: 6469250 extra: 554696706
RMENU           WM_KEYDOWN     : vkCode  a5 scanCode   0 flags:  16, time: 6469281 extra: 554696706
RMENU           WM_KEYDOWN     : vkCode  a5 scanCode   0 flags:  16, time: 6469312 extra: 554696706
RMENU           WM_KEYDOWN     : vkCode  a5 scanCode   0 flags:  16, time: 6469343 extra: 554696706
RMENU           WM_KEYDOWN     : vkCode  a5 scanCode   0 flags:  16, time: 6469375 extra: 554696706
RMENU           WM_KEYDOWN     : vkCode  a5 scanCode   0 flags:  16, time: 6469421 extra: 554696706
RMENU           WM_KEYDOWN     : vkCode  a5 scanCode   0 flags:  16, time: 6469453 extra: 554696706
RMENU           WM_KEYDOWN     : vkCode  a5 scanCode   0 flags:  16, time: 6469484 extra: 554696706
RMENU           WM_KEYDOWN     : vkCode  a5 scanCode   0 flags:  16, time: 6469515 extra: 554696706
OEM_4           WM_KEYDOWN     : vkCode  db scanCode   0 flags:  16, time: 6469515 extra: 554696706
OEM_4           WM_KEYUP       : vkCode  db scanCode   0 flags: 144, time: 6469718 extra: 554696706
RMENU           WM_KEYUP       : vkCode  a5 scanCode   0 flags: 144, time: 6470312 extra: 554696706

the leading LCONTROL WM_KEYDOWN in the previous debug block seems off to me, as I have definitely not pressed <LCONTROL> in order to create a BACKSLASH.

After having pressed BACKSLASH, I see (as described further above in this thread) that the Neovim commandline closes. Instead of printing a BACKSLASH.

Now, I want to test whether or not I am still able to switch applications by pressing the ordinary <L-ALT> + <TAB>. And I see, while pressing these keys, applications don't switch...

LMENU           WM_KEYDOWN     : vkCode  a4 scanCode   0 flags:  16, time: 6477359 extra: 554696706
TAB             WM_KEYDOWN     : vkCode   9 scanCode   0 flags:  16, time: 6477656 extra: 554696706
TAB             WM_KEYUP       : vkCode   9 scanCode   0 flags: 144, time: 6477781 extra: 554696706
LMENU           WM_KEYUP       : vkCode  a4 scanCode   0 flags: 144, time: 6478500 extra: 554696706
LMENU           WM_KEYDOWN     : vkCode  a4 scanCode   0 flags:  16, time: 6479562 extra: 554696706
TAB             WM_KEYDOWN     : vkCode   9 scanCode   0 flags:  16, time: 6479656 extra: 554696706
TAB             WM_KEYUP       : vkCode   9 scanCode   0 flags: 144, time: 6479796 extra: 554696706
LMENU           WM_KEYUP       : vkCode  a4 scanCode   0 flags: 144, time: 6480078 extra: 554696706
LMENU           WM_KEYDOWN     : vkCode  a4 scanCode   0 flags:  16, time: 6480562 extra: 554696706
TAB             WM_KEYDOWN     : vkCode   9 scanCode   0 flags:  16, time: 6480750 extra: 554696706
TAB             WM_KEYUP       : vkCode   9 scanCode   0 flags: 144, time: 6480875 extra: 554696706
LMENU           WM_KEYUP       : vkCode  a4 scanCode   0 flags: 144, time: 6481015 extra: 554696706
LMENU           WM_KEYDOWN     : vkCode  a4 scanCode   0 flags:  16, time: 6481187 extra: 554696706
TAB             WM_KEYDOWN     : vkCode   9 scanCode   0 flags:  16, time: 6481343 extra: 554696706
TAB             WM_KEYUP       : vkCode   9 scanCode   0 flags: 144, time: 6481453 extra: 554696706
TAB             WM_KEYDOWN     : vkCode   9 scanCode   0 flags:  16, time: 6481609 extra: 554696706
TAB             WM_KEYUP       : vkCode   9 scanCode   0 flags: 144, time: 6481734 extra: 554696706
TAB             WM_KEYDOWN     : vkCode   9 scanCode   0 flags:  16, time: 6481890 extra: 554696706
TAB             WM_KEYUP       : vkCode   9 scanCode   0 flags: 144, time: 6482000 extra: 554696706
TAB             WM_KEYDOWN     : vkCode   9 scanCode   0 flags:  16, time: 6482203 extra: 554696706
TAB             WM_KEYUP       : vkCode   9 scanCode   0 flags: 144, time: 6482312 extra: 554696706
TAB             WM_KEYDOWN     : vkCode   9 scanCode   0 flags:  16, time: 6482515 extra: 554696706
TAB             WM_KEYUP       : vkCode   9 scanCode   0 flags: 144, time: 6482625 extra: 554696706
LMENU           WM_KEYUP       : vkCode  a4 scanCode   0 flags: 144, time: 6482703 extra: 554696706

Instead of switching applications, the start menu has opened. thus I press <ESC> to close it...

ESCAPE          WM_KEYDOWN     : vkCode  1b scanCode   0 flags:  16, time: 6486750 extra: 554696706
ESCAPE          WM_KEYUP       : vkCode  1b scanCode   0 flags: 144, time: 6486796 extra: 554696706
ESCAPE          WM_KEYDOWN     : vkCode  1b scanCode   0 flags:  16, time: 6488109 extra: 554696706
ESCAPE          WM_KEYUP       : vkCode  1b scanCode   0 flags: 144, time: 6488203 extra: 554696706

Now, I press <LCONTROL>, then <LWIN>, then <L-ALT>, ... and some variations of it in order to get the application switchin to run again...

LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6488953 extra: 554696706
LCONTROL        WM_KEYUP       : vkCode  a2 scanCode   0 flags: 144, time: 6489093 extra: 554696706
LWIN            WM_KEYDOWN     : vkCode  5b scanCode   0 flags:  16, time: 6489250 extra: 554696706
LWIN            WM_KEYUP       : vkCode  5b scanCode   0 flags: 144, time: 6489375 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6489484 extra: 554696706
LMENU           WM_KEYUP       : vkCode  a4 scanCode   0 flags: 144, time: 6489609 extra: 554696706
LWIN            WM_KEYDOWN     : vkCode  5b scanCode   0 flags:  16, time: 6490062 extra: 554696706
LWIN            WM_KEYUP       : vkCode  5b scanCode   0 flags: 144, time: 6490187 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6490281 extra: 554696706
LCONTROL        WM_KEYUP       : vkCode  a2 scanCode   0 flags: 144, time: 6490421 extra: 554696706
LWIN            WM_KEYDOWN     : vkCode  5b scanCode   0 flags:  16, time: 6490593 extra: 554696706
LWIN            WM_KEYUP       : vkCode  5b scanCode   0 flags: 144, time: 6490734 extra: 554696706
LWIN            WM_KEYDOWN     : vkCode  5b scanCode   0 flags:  16, time: 6491187 extra: 554696706
LWIN            WM_KEYUP       : vkCode  5b scanCode   0 flags: 144, time: 6491312 extra: 554696706

Now, I try switching applications using <L-ALT> + <TAB>, and it works properly again. applications get switched...

LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6492062 extra: 554696706
TAB             WM_SYSKEYDOWN  : vkCode   9 scanCode   0 flags:  48, time: 6492437 extra: 554696706
TAB             WM_SYSKEYUP    : vkCode   9 scanCode   0 flags: 176, time: 6492531 extra: 554696706
LMENU           WM_KEYUP       : vkCode  a4 scanCode   0 flags: 144, time: 6493125 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6493515 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494015 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494046 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494078 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494109 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494140 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494187 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494218 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494250 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494281 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494312 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494343 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494375 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494406 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494437 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494484 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494515 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494546 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494578 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494609 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494640 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494671 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494703 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494750 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6494781 extra: 554696706
TAB             WM_SYSKEYDOWN  : vkCode   9 scanCode   0 flags:  48, time: 6494796 extra: 554696706
TAB             WM_SYSKEYUP    : vkCode   9 scanCode   0 flags: 176, time: 6494906 extra: 554696706
TAB             WM_SYSKEYDOWN  : vkCode   9 scanCode   0 flags:  48, time: 6495234 extra: 554696706
TAB             WM_SYSKEYUP    : vkCode   9 scanCode   0 flags: 176, time: 6495359 extra: 554696706
LMENU           WM_KEYUP       : vkCode  a4 scanCode   0 flags: 144, time: 6495500 extra: 554696706
LMENU           WM_SYSKEYDOWN  : vkCode  a4 scanCode   0 flags:  48, time: 6497296 extra: 554696706
TAB             WM_SYSKEYDOWN  : vkCode   9 scanCode   0 flags:  48, time: 6497468 extra: 554696706
TAB             WM_SYSKEYUP    : vkCode   9 scanCode   0 flags: 176, time: 6497625 extra: 554696706
LMENU           WM_KEYUP       : vkCode  a4 scanCode   0 flags: 144, time: 6498546 extra: 554696706

having reached the terminal where I started wm.pyw, I can now send a <CTRL> + <C> to end the process...

LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6500187 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6500687 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6500734 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6500765 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6500796 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6500828 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6500859 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6500890 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6500921 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6500953 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6501000 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6501031 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6501062 extra: 554696706
LCONTROL        WM_KEYDOWN     : vkCode  a2 scanCode   0 flags:  16, time: 6501093 extra: 554696706
C               WM_KEYDOWN     : vkCode  43 scanCode   0 flags:  16, time: 6501109 extra: 554696706
C               WM_KEYUP       : vkCode  43 scanCode  2e flags: 128, time: 6501375 extra: 0
LCONTROL        WM_KEYUP       : vkCode  a2 scanCode  1d flags: 128, time: 6501671 extra: 0

That's all.

Aug 20 '23 11:08 nnako

Thanks, @nnako , very thorough... The unexpected LControl from the 3rd step caused all the mess IMO.

However, I don't have a clue why it happened yet, can it be reproduced stably? What if you don't enter the ASDF, will it produces the unexpected LControl again?

PS, do you mind posting a picture of your keyboard? thanks.

Aug 21 '23 06:08 klesh

Hi @klesh , yes, the <LControl> seems odd, here. And odd is also that it appears before long pressing the <RMenu> prior to the <BACKSLASH>. So, it does not even seem to have anything to do with pressing the <BACKSLASH> key but with something preceeding it.

here my keyboard (Keychron K8 PRO German):

I just checked the low-level keyboard interface WITHOUT further pyw file by opening the standard console, walking into the project folder and issuing py -m jigsawwm.w32.hook. This is what I get for the specific key presses:

for a capital "L" I press <SHIFT> + <L> . In the console I get...

RSHIFT          WM_KEYDOWN     : vkCode  a1 scanCode  36 flags:   1, time: 29816906 extra: 0
L               WM_KEYDOWN     : vkCode  4c scanCode  26 flags:   0, time: 29817031 extra: 0
L               WM_KEYUP       : vkCode  4c scanCode  26 flags: 128, time: 29817234 extra: 0
RSHIFT          WM_KEYUP       : vkCode  a1 scanCode  36 flags: 129, time: 29817281 extra: 0

for a "BACKSLASH" I press <R-ALT> + <SZLIG> as seen before...

LCONTROL        WM_SYSKEYDOWN  : vkCode  a2 scanCode 21d flags:  32, time: 29490734 extra: 0
RMENU           WM_SYSKEYDOWN  : vkCode  a5 scanCode  38 flags:  33, time: 29490734 extra: 0
OEM_4           WM_SYSKEYDOWN  : vkCode  db scanCode   c flags:  32, time: 29491171 extra: 0
OEM_4           WM_SYSKEYUP    : vkCode  db scanCode   c flags: 160, time: 29491281 extra: 0
LCONTROL        WM_KEYUP       : vkCode  a2 scanCode 21d flags: 128, time: 29491546 extra: 0
RMENU           WM_KEYUP       : vkCode  a5 scanCode  38 flags: 129, time: 29491546 extra: 0

for a "[]" (opening and closing square brackets) I hold <R-ALT> and press <8> then <9>...

LCONTROL        WM_SYSKEYDOWN  : vkCode  a2 scanCode 21d flags:  32, time: 29649187 extra: 0
RMENU           WM_SYSKEYDOWN  : vkCode  a5 scanCode  38 flags:  33, time: 29649187 extra: 0
KEY_8           WM_SYSKEYDOWN  : vkCode  38 scanCode   9 flags:  32, time: 29649578 extra: 0
KEY_8           WM_SYSKEYUP    : vkCode  38 scanCode   9 flags: 160, time: 29649656 extra: 0
KEY_9           WM_SYSKEYDOWN  : vkCode  39 scanCode   a flags:  32, time: 29649906 extra: 0
KEY_9           WM_SYSKEYUP    : vkCode  39 scanCode   a flags: 160, time: 29650046 extra: 0
LCONTROL        WM_KEYUP       : vkCode  a2 scanCode 21d flags: 128, time: 29650218 extra: 0
RMENU           WM_KEYUP       : vkCode  a5 scanCode  38 flags: 129, time: 29650218 extra: 0

when I press and release the single key <R-ALT>, I see...

LCONTROL        WM_SYSKEYDOWN  : vkCode  a2 scanCode 21d flags:  32, time: 30392812 extra: 0
RMENU           WM_SYSKEYDOWN  : vkCode  a5 scanCode  38 flags:  33, time: 30392812 extra: 0
LCONTROL        WM_KEYUP       : vkCode  a2 scanCode 21d flags: 128, time: 30393078 extra: 0
RMENU           WM_KEYUP       : vkCode  a5 scanCode  38 flags: 129, time: 30393078 extra: 0

So, the problem does not seem to have a relation to "BACKSLASH" at all. But rather the <R-ALT> key:

it misteriously triggers a LCONTROL down, then a RMENU down (which seems to be the <R-ALT>)
when releasing all keys, the release order for LCONTROL and RMENU seems inverted, here
and in my former example, LCONTROL was not even released.

Aug 21 '23 14:08 nnako

According to https://en.wikipedia.org/wiki/AltGr_key#Ctrl+Alt , AltGr is an equivalent of Ctrl + Alt, I think it makes sense for the console to output sth like that. So, if i understand it correctly:

\ and [] can be entered as expected when typing casually.
LControl was not released when typing fast with ASDF before entering \ or any other symbols that involve AltGr

Am I correct?

PS: nice keyboard there.

Aug 22 '23 06:08 klesh

Interesting.

\ and [] can be entered as expected when typing casually

yes, those keys are presented on the text input when generated using <ALTGR> with the appropriate key.

LControl was not released when typing fast with ASDF before entering \ or any other symbols that involve <AltGr>

correct. LControl seems to have been swallowed, looking at the keyboard debug trace.

I just tried to generate \ by pressing <LCONTROL> + <ALT> + <SZLIG> (not using <ALTGR>), as seen in the link you posted about alternative key presses for <ALTGR>. I see that \ gets NOT displayed. So, it seems that <LCONTROL> + <ALT> is not a reliable replacement for <ALTGR>. If the low-level keyboard implementation could forward the "real" <ALTGR>, not the alternative <LCONTROL> + <ALT> the problem might be solved.

What do you think?

Aug 22 '23 08:08 nnako

@nnako The way I see it , the AltGr events are being forwarded correctly so \ and [] popped out in your input textbox without problem. To clarify, the low-level keyboard implementation doesn't do anything for AltGr events, in fact, whatever you saw on the console were the events that hook.py received from the OS, in other words, the OS generates the following events when you press AltGr

LCONTROL        WM_SYSKEYDOWN  : vkCode  a2 scanCode 21d flags:  32, time: 30392812 extra: 0
RMENU           WM_SYSKEYDOWN  : vkCode  a5 scanCode  38 flags:  33, time: 30392812 extra: 0
LCONTROL        WM_KEYUP       : vkCode  a2 scanCode 21d flags: 128, time: 30393078 extra: 0
RMENU           WM_KEYUP       : vkCode  a5 scanCode  38 flags: 129, time: 30393078 extra: 0

I think it was caused by the F(LControl when held) and AltGr being pressed simultaneously. Can you reproduce the phenomenon stably?

Aug 22 '23 09:08 klesh

Hi @klesh ,

The way I see it , the AltGr events are being forwarded correctly so \ and [] popped out in your input textbox without problem.

This is only the case when I haven't started wm.pyw. So, in normal operation (without interception by JigsawWM), the keys \ and [] appear as usual within whatever currently active text editor. Even when the low-level key debugger (started in some terminal via py -m jigsawwm.w32.hook is running, the "normal" behaviour is seen. But as soon as I start JigsawWM (in my case via python .\examples\wm.pyw, the chars \ and [] don't appear on the input row as seen in the screen recordings. And the WM_KEYUP event for LCONTROL is missing (not visible in the key debugger running in parallel) after pressing and releasing any key and <ALTGR>.

here another video:

https://github.com/klesh/JigsawWM/assets/5845602/ed9f7b95-c306-4fce-b092-0e4cc95b9066

Aug 22 '23 10:08 nnako

Hi @klesh ,

The way I see it , the AltGr events are being forwarded correctly so \ and [] popped out in your input textbox without problem.

This is only the case when I haven't started wm.pyw. So, in normal operation (without interception by JigsawWM), the keys \ and [] appear as usual within whatever currently active text editor. Even when the low-level key debugger (started in some terminal via py -m jigsawwm.w32.hook is running, the "normal" behaviour is seen. But as soon as I start JigsawWM (in my case via python .\examples\wm.pyw, the chars \ and [] don't appear on the input row as seen in the screen recordings. And the WM_KEYUP event for LCONTROL is missing (not visible in the key debugger running in parallel) after pressing and releasing any key and <ALTGR>.

here another video:

odd_keyboard_behavior_in_neovim_app__02__rf32.mp4

Aah, I see. Thanks for the clarification. I just updated the main branch, please see if it fixes your problem.

Aug 23 '23 01:08 klesh

Hi @klesh , thanks for the fix.

Unfortunately, it does not yet work quite well, as a LCONTROL seems to still be halted (active). Please see the new video:

https://github.com/klesh/JigsawWM/assets/5845602/b06b4c1e-a4aa-49c1-b0bc-ad62046debae

Some further explanation of the effect that is seen on the video:

after pressing <ALTGR> + <SZLIG>, the correct BACKSLASH character is created within the editor
the debugging states an additional LCONTROL KEYUP, which seems odd as it was already said two debug lines before. I would (for the visible effect) have at least expected a LCONTROL KEYDOWN.
the LCONTROL is still pressed, though, as seen in the shift operations within the editor
after pressing and releasing the <WIN> key, the halted LCONTROL seems to be healed again

Aug 23 '23 07:08 nnako

yeah, the extra LControl UP event is undesirable, I can reproduce it on my PC, seems like the OS would produce an extra LControl UP event for whatever reason. However, an extra UP event should not be causing any problems... weird 😂

Aug 23 '23 08:08 klesh

Hi @klesh , could it be that this was a real operating system problem? I recall these hanging CONTROL keys from other issues within other similar projects (e.g. within the "keyboard" module). In my opinion, we (on the non-system-development side) would only have a few options:

spot the cause of our issue as close as possible and then hand it over to Microsoft's support ;-)
for the time being, try to provide a workaround by identifying this hanging CONTROL key (maybe, there is a list of active keys where this hanging key would be visible?), make it plausible as actually "hanging", and then neutralize it (I achieved it by pressing <WIN>).
try to figure out what workaround other projects have found to cope with it. There is a project named ahk which is a wrapper around the AutoHotKey project, thus providing a Python interface as a replacement for the "cryptic" AHK syntax of the original project. I tried that one some weeks ago and I haven't encountered our issue concerning "hanging keys", yet.

what do you think?

Aug 23 '23 16:08 nnako

@nnako Yup, they all make sense to me... The problem is that I couldn't reproduce the hanging Control key on my PC with German keyboard layout... I need a stable way to reproduce the bug before I can fix it.

Aug 24 '23 13:08 klesh

Maybe, I could try to fix it on my side. But I would need your help in navigating me through your code until I have gained some confidence with it. Please see my attached structure diagram. It tries to describe "how the information flows"

from the physical keyboard
through the operating system layers (visible as OS events for us)
then through your source code files
until their end points (applications like text editors)

As soon as this diagram resembles the "real" world, fixing those problems would become much easier for me, I guess.

Aug 24 '23 22:08 nnako

Nice, you got it pretty much correct except for the jmk module, it is more like the following:

SystemInput intercepts related events and sends them to the JmkCore
JmkCore interprets and transforms events based on the configuration (TapHold and Key, etc.) and sends them to the JmkHotkey
JmkHotkey checks if the registered hotkeys should be fired or redirect events to the SystemOutput
SystemOutput simply calls sendinput to send kbd/mouse events to the OS

Aug 25 '23 03:08 klesh

Hi @klesh , thanks for your feedback. I updated the drawing a bit. Please note the following:

the abstraction level is a bit higher than the class level, so that the technical implementation is only structured using the source files. not the classes within files. I think, it makes the first-understanding without too many details much easier than class diagrams.
the blue arrow heads tell the reader where information is "generated" or "actively read" / polled. so, they are located at tools, scripts, filters and hardware components.
the green connection lines show where user information (in our case, keyboard events) flow.
the dashed black lines carry other trigger or configuration information between the participants.

Aug 26 '23 13:08 nnako

Yup, it looks good to me.

Aug 28 '23 13:08 klesh