jtroo
Feature request: Auto Mouse Keys layer
Is your feature request related to a problem? Please describe.
when I move the mouse cursor using my left hand on the trackpoint, I would like mouse buttons to appear under my right hand fingers on my laptop keyboard. after a timeout, I would like the keys to revert. the generalization of this is automousekeys layer switching.
previously, I've been using mannaharbour's automousekeys implementation with QMK-powered external keyboards.
Describe the solution you'd like.
My initial thought for an implementation was a defcfg feature flag that takes a layer name and a timeout. a very nice-to-have would be to also accept a list of layers upon which the automousekeys switching can activate. I might like to only have it on my base layer, but not on my navigation layer, for example.
however, I wasn't sure how to force layer switches, and especially how to return from one, with keyberon.
Describe alternatives you've considered.
I implemented a Proof of Concept in #1080. This takes an easier approach of injecting key press and release events for an unused key which the user is expected to map to a layer hold. For example:
(defsrc
k l ;
F24
)
(deflayer qwerty
k l ;
layer-while-held mouse
)
(deflayer mouse
mlft mmid mrgt
XX
)
The addition of a key that doesn't exist on the keyboard seems clunky, but it does provide a nice way to manage the per-layer configuration of the feature. I'm half convinced this is an OK way to handle it, but I think the initial solution is cleaner overall.
Additional context
I feel the layer switching needs to be a bit smarter to be fully useful.
In the QMK version I update the timestamp when mouse key events occur, as well. for this version I might like to ignore real mouse keys (if I'm using the physical buttons I don't need the layer switch) but update the timestamp when non-transparent key events occur on the mouse layer. not sure how to get that info.
For switching the mouse layer off, I would really like this to occur any time I press a transparent key on the layer, and I want to trigger the underlying key mapping as well. I think any press of an unmapped key would also ideally switch the mouse layer off as this is also a sign the user has resumed typing.
finally, since I also use homerow mods, I would like to try having hold behaviors on the mouse layer trigger a layer release as well.
Sounds like you might be able to achieve this today with some combination of:
Is there a way to detect if a finger or fingers are on the touchpad (even when not moving)? If so, that would be great in addition to this functionality.
this is the config I've been using. the basic functionality is fine, but the "disable mouse layer when typing" is sticky. frequently the mouse layer won't re-enable after an early exit. this issue was much worse when I was using release-layer, but still persists with this release-vkey approach.
(defcfg
process-unmapped-keys yes
)
(defsrc
grv 1 2 3 4 5 6 7 8 9 0 - = bspc
tab q w e r t y u i o p [ ] \
caps a s d f g h j k l ; ' ret
lsft z x c v b n m , . / rsft
lctl lmet lalt spc ralt prnt rctl
F24
)
(defvirtualkeys
mouse (layer-while-held mouse-wide)
)
(defalias
;;msk (layer-while-held mouse-wide)
mon (on-press press-vkey mouse)
moff (on-press release-vkey mouse)
moffr (on-release release-vkey mouse)
midle (on-idle 1000 release-vkey mouse)
msk (multi
@mon
@moffr
;;@midle
)
;;_ (switch
;; ((layer mouse-wide)) (release-key F24) fallthrough
;; () _ break
;; )
_ (multi
;;(release-layer mouse-wide)
@moff
;;(release-key F24)
_
)
)
(deflayer qwerty
grv 1 2 3 4 5 6 7 8 9 0 - = bspc
tab q w e r t y u i o p [ ] \
bspc a s d f g h j k l ; ' ret
lsft z x c v b n m , . / rsft
lctl lmet lalt spc ralt prnt rctl
@msk
)
(deflayer mouse-wide
@_ @_ @_ @_ @_ @_ @_ @_ @_ @_ @_ @_ @_ @_
@_ @_ @_ @_ @_ @_ @_ @_ @_ @_ @_ @_ @_ @_
@_ @_ @_ @_ @_ @_ @_ @_ @_ @_ @_ @_ @_
@_ @_ @_ @_ @_ @_ @_ @_ mlft mmid mrgt @_
@_ @_ @_ @_ @_ @_ @_
XX
)
There is more recent functionality that might help with this: hold-for-duration.
Any update on this issue? I have been using the proof of concept fork for awhile now as I find this feature a necessity with my thinkpad keyboard, but there are some bugs with it the implementation and I would love for this to be in the official release. I would be happy to try to implement this myself as well, but was wondering if you might have any thoughts on how best to do it?
thanks for commenting. I wasn't sure if anyone else actually found it useful. I'm stuck right now because when the layer is disabled by typing it's reactivation is unreliable. since its unreliable I don't use it, and so I'm not reminded to work on a fix.
what issues are you having? without a keymap that sends F24 key releases it should be dependable as long as you wait for the timeout before switching to typing. If you are on linux I recommend using https://github.com/Airblader/unclutter-xfixes with the timeout matched to the automousekeys timeout, and a red cursor theme like redglass. if you see the cursor, don't start typing yet :)
the biggest thing I need to move forward is to collect some debug logs to see why the layer isn't reactivated after being deactivated by typing.
somewhat unrelated, but:
i thought it would be nice to use the capslock LED to indicate that the automouselayer is active. however once kanata captures the keyboard, I can no longer toggle the LED by echoing to the proper file in /sys/. Maybe there is a way to control the LED inside kanata?
EDIT: maybe really unrelated, but I also sometimes manage to hit capslock before kanata starts and then the LED remains lit no matter what I do, so that might be another good reason to hook into LED state.
i thought it would be nice to use the capslock LED to indicate that the automouselayer is active. however once kanata captures the keyboard, I can no longer toggle the LED by echoing to the proper file in /sys/. Maybe there is a way to control the LED inside kanata?
I've been having a similar idea on and off for some time now - light up screen borders in some noticable color (e.g. red) when a specific layer is active (i want that for mouse layer). But I never actually got to code it up, partly because I'm probably not gonna like the result - just like with kanata-tray (full disclosure that I'm the author), which shows current active layer in tray icon. You might want to check it instead of a the led indicator idea.
Anyways that's getting off topic, so yeah. Just wanted to mention it.
thanks for commenting. I wasn't sure if anyone else actually found it useful. I'm stuck right now because when the layer is disabled by typing it's reactivation is unreliable. since its unreliable I don't use it, and so I'm not reminded to work on a fix.
Funny, I had the idea since there aren't the physical mouse buttons on this thinkpad and came across this issue and your pull request and so I started using kanata because of that. The comfort of having the buttons on my home row has actually made the trackpoint something I think I can't live without now so I keep using this despite the bugs.
what issues are you having? without a keymap that sends F24 key releases it should be dependable as long as you wait for the timeout before switching to typing. If you are on linux I recommend using https://github.com/Airblader/unclutter-xfixes with the timeout matched to the automousekeys timeout, and a red cursor theme like redglass. if you see the cursor, don't start typing yet :)
I haven't actually spent time to look into logs of my issues yet, but what I have noticed is what you are saying, which is unreliable reactivation of the layer, as well as kanata occasionally crashing when this happens.
After a little bit of trying I was able to produce a log of a crash, if a longer or more detailed one helps let me know:
Apr 02 19:10:15 thinkpad kanata[92899]: 19:10:15.6204 [INFO] Mouse Layer Off
Apr 02 19:10:15 thinkpad kanata[92899]: 19:10:15.6369 [INFO] Mouse Layer On
Apr 02 19:10:19 thinkpad kanata[92899]: 19:10:19.3105 [INFO] Mouse Layer Off
Apr 02 19:10:19 thinkpad kanata[92899]: 19:10:19.4414 [INFO] Entered layer:
Apr 02 19:10:19 thinkpad kanata[92899]: (deflayer base
Apr 02 19:10:19 thinkpad kanata[92899]: caps _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Apr 02 19:10:19 thinkpad kanata[92899]: _ _ _ _ _ _ _ _ _ _ _ _ _ _
Apr 02 19:10:19 thinkpad kanata[92899]: _ _ _ _ _ _ _ _ _ _ _ _ _ _
Apr 02 19:10:19 thinkpad kanata[92899]: @cap @a @s @d @f _ _ @j @k @l @; _ _
Apr 02 19:10:19 thinkpad kanata[92899]: _ _ _ _ _ _ _ _ _ _ _ _
Apr 02 19:10:19 thinkpad kanata[92899]: @wkf _ lalt lmet @spm rmet _ _ _ _ _
Apr 02 19:10:19 thinkpad kanata[92899]: _ _ _
Apr 02 19:10:19 thinkpad kanata[92899]: @tp
Apr 02 19:10:19 thinkpad kanata[92899]: )
Apr 02 19:10:19 thinkpad kanata[92899]: 19:10:19.6143 [INFO] Mouse Layer On
Apr 02 19:10:19 thinkpad kanata[92899]: 19:10:19.6144 [INFO] Entered layer:
Apr 02 19:10:19 thinkpad kanata[92899]: (deflayer trackpoint
Apr 02 19:10:19 thinkpad kanata[92899]: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Apr 02 19:10:19 thinkpad kanata[92899]: _ _ _ _ _ _ _ _ _ _ _ _ _ _
Apr 02 19:10:19 thinkpad kanata[92899]: _ _ _ _ _ _ _ _ _ _ _ _ _ _
Apr 02 19:10:19 thinkpad kanata[92899]: _ _ _ _ _ _ _ mlft mrgt _ mmid _ _
Apr 02 19:10:19 thinkpad kanata[92899]: _ _ _ _ _ _ _ _ _ _ _ _
Apr 02 19:10:19 thinkpad kanata[92899]: _ _ _ _ _ _ _ _ _ _ _
Apr 02 19:10:19 thinkpad kanata[92899]: _ _ _
Apr 02 19:10:19 thinkpad kanata[92899]: XX
Apr 02 19:10:19 thinkpad kanata[92899]: )
Apr 02 19:10:24 thinkpad kanata[92899]: 19:10:24.6328 [ERROR] second time provided was later than self
Apr 02 19:10:24 thinkpad kanata[92899]: Press enter to exit
Apr 02 19:10:24 thinkpad kanata[92899]: Error: second time provided was later than self
Apr 02 19:10:24 thinkpad kanata[92899]: 19:10:24.6329 [ERROR] channel disconnected
Apr 02 19:10:24 thinkpad systemd[1]: kanata.service: Main process exited, code=exited, status=1/FAILURE
Apr 02 19:10:24 thinkpad systemd[1]: kanata.service: Failed with result 'exit-code'.
Apr 02 19:10:24 thinkpad systemd[1]: kanata.service: Consumed 11.042s CPU time, 5M memory peak.
Would (multi (on-press release-vkey mouse) mlft) help? Not sure about dblclick
Interesting. There's either a bug in Kanata where the code is comparing values incorrectly; which I have never reproduced; or your system has a bug in the monotonic time implementation.
Apr 02 19:10:24 thinkpad kanata[92899]: 19:10:24.6328 [ERROR] second time provided was later than self
I added the -t flag to my service for the next time it crashes but I haven't been able to reproduce it yet today.
Timedatectl shows this:
Local time: Thu 2025-04-03 13:10:38 PDT
Universal time: Thu 2025-04-03 20:10:38 UTC
RTC time: Thu 2025-04-03 20:10:39
Time zone: America/Los_Angeles (PDT, -0700)
System clock synchronized: no
NTP service: inactive
RTC in local TZ: no
I am on Arch linux without any special time synchronization configured. Not sure if this is what you meant by my systems monotonic time.
I definitely haven't seen any crashes that I'm aware of. the only time comparison I've added is here https://github.com/jtroo/kanata/pull/1080/files#diff-428b2d4ad36aba907ce64670e822d08895db03b69b73f826d6155f218c114092R74-R75 and it doesn't directly compare 2 timestamps, but rather a time and a duration. under the hood it must take a current time to add to the duration, and add an assertion about monotonic time.
do crashes happen after suspending or hibernation your laptop? that is one way that time can go awry.
I think it makes sense to use the SystemTime timestamp of the event, rather than timestamping when kanata gets around to processing it. functionally, the only difference seems to be that Instant conceals the Error that SystemTime throws during comparison if the current time is older than the saved timestamp.
Instead I pushed a new commit that catches and ignores the Error condition.
Unless there's a good reason, you should default to using monotonic time (instant) to measure durations. System time suffers from things like ntp corrections and any number of things that might change the OS system time.
fair enough. I implemented a new version to go with your sugestion to use hold-for-duration. this works perfectly and is a simpler implementation without added timestamp tracking.
https://github.com/wolfwood/kanata/tree/feature/automousekeys-simple
only issue is I don't understand why it works. specifically, why after the mouse layer is disabled by typing on a transparent key, continued mouse cursor movement doesn't re-enable the mouse layer, until movement stops for the allotted time period and restarts. this is exactly the behavior I want, but if feels like a bug :)
example config
(defsrc)
(defvirtualkeys
mouse (layer-while-held mouse-layer)
)
(defalias
mhld (hold-for-duration 750 mouse)
moff (on-press release-vkey mouse)
_ (multi
@moff
_
)
)
(deflayermap (base)
F24 @mhld
)
;; places mouse keys on the row above the home row.
;; pressing any other keys exits the mouse layer until mouse movement stops and restarts again.
(deflayermap (mouse-layer)
u mlft
i mmid
o mrgt
F24 @mhld
___ @_
)
my code just sends a tap event for F24 every time there is a RelAxis event.
I guess hold-for-duration isn't actually aware that the virtual key has been released, so its timer is just reset, rather than triggering the press event again. makes sense.
I fixed the old PR to use Instant for posterity, but I have opened a new PR #1598 expanding on my earlier simplified approach which is far superior.
using hold-for-duration allows, for example, left click to also extend the timer, in addition to cursor movement, in order to give extra time to complete a double click. I think this demonstrates the value in implementing the mouse keys layer (mostly) at the config-level.
I tried to implement this for windows (interception) as well but don't have an easy way to test at the moment.
I added the defcfg parameter mouse-movement-key which specifies a key to tap on every mouse cursor event. I also wanted to add a key for the purpose, mvmt although I just pointed it at KEY_766. renaming the enum broke tests and I wasn't sure if it was safe to grow the list of enums. it should be possible to use eg nop0 instead, but something descriptive seems nice.
Awesome, I like the new approach with the control it allows from the configuration side! I have been using the commit you shared 4 days ago from the simple mouse keys branch and have had no issues so far, going to update to the PR soon so I can use the mvmt key instead of F24
one thing I haven't figured out.
I use homerow mods, and right now I don't have a way to Shift-left click or Ctrl-left click, because holding mods exits my mouse layer.
maybe I need to make new tap-hold actions that exit the mouse layer and fall through on a tap, but act as a mod on a hold? don't know if I would prefer to hold with my trackpoint hand or clicking hand.
or maybe I need combos like middle+left click for a shift left click and right+left for ctrl left click. less flexible than separate mod keys but also less hand contortion.
I just had this issue today myself, wanting to use Hyprlands super + mouse click window dragging. I personally use my trackpoint and click with the same hand (using j and k for mouse clicks), so it made sense to just put my home row mods on my trackpoint layer for my left hand. I think to address letting them tap through the mouse layer additional tap-hold aliases might be the best way, for my config I had to add the mouse-layer off action in between the key press and my typing layer action in my homerow tap like this:
tpa (tap-hold 200 200 (multi a @tpoff @tap) lalt)
tps (tap-hold 200 170 (multi s @tpoff @tap) lsft)
tpd (tap-hold 200 170 (multi d @tpoff @tap) lmet)
tpf (tap-hold 200 150 (multi f @tpoff @tap) lctl)
I had a hope that I could give myself extra time for double clicking without keeping the mouse layer active longer by instead having my base layer u key check if the last typed key was a mouse click and, if it hasn't been too long, continuing to click. however this doesn't seem to work.
u (switch
((and (key-history mlft 1) (key-timing 1 lt 400))) mlft break
() u break
)
my best guess is that the history is a keyberon feature and since keyberon doesn't support mouse keys, the history check always fails.
I'm wondering if it would be possible to feed keycodes with no output mapping into keyberon to populate the history?
Keyberon does support mouse keys but the history is likely overridden by the mouse movement history.