zmkfirmware
Add PS/2 Mouse / Trackpoint / Trackpad / Trackball support
Hey guys, this PR adds support for PS2 Mouse Devices.
Note:
This PR has recently been updated with a new driver that uses the UART chip to process the PS/2 signals, which leads to MUCH better performance.
If you tried the driver before and had issue, please try the new version.
The instructions below have been updated for the new version.
1. How to test
I have recently converted this PR into a standalone zmk module that you can use to add PS/2 mouse & trackpoint support to your keyboard.
You can find it here: https://github.com/infused-kim/kb_zmk_ps2_mouse_trackpoint_driver
The code here won't be updated anymore until we get closer to the stage where it may get merged into zmk main.
2. Features
In addition to supporting all PS/2 pointing devices, the driver has a few noteworthy features...
2.1. Great performance through the use of the UART chip
PS/2 is a very old protocol and unfortunately not used in many devices anymore (apart from TrackPoints). Therefore most microcontrollers don't include hardware chips that handle the communication with PS/2 devices (unlike, for example, SPI).
TrackPoints send data at a rate of almost 15,000 times per second and unfortunately, the most popular controllers in the zmk community are not able to keep up with the communication, at this speed, when you process it using GPIO interrupts.
However, I found a way to take advantage of the UART hardware chip to process the data. This ensures great performance even without proper PS/2 hardware support on the MCU.
If you are interested in how this works, you can read more about it in the development journey section.
2.2. Automatic Layer Toggling on mouse movement
You can also automatically toggle a layer whenever there is mouse movement. This way you can turn your thumb cluster keys into mouse buttons whenever you touch the TrackPoint.
2.3. TrackPoint related features
2.3.1. TrackPoint Configuration with key behaviors
TrackPoints have various settings that can change the algorithm for the mouse movement, but recompiling and reflashing your keyboard to test them is very tedious and makes comparing settings very hard.
So, I added the ability to adjust these settings at run-time through key behaviors, such as &mms MS_TP_SENSITIVITY_INCR. This way you can easily find the right settings for your keyboard and preference.
You can adjust the following settings:
- Sensitivity
- Negative Inertia
- Transfer Function Upper Plateau Speed (value6)
- The pressure sensitivity for press-to-select
Read the TrackPoint System Version 4.0 Engineering Specification to better understand how these settings affect the mouse behavior.
And check out the instructions in the example zmk-config to see how to use the settings key layer.
2.3.2. Power-On-Reset Support
Many TrackPoints require an additional reset pin. Most people in the ergo keyboard community create a reset circuit that uses resistors and capacitors to implement the functionality.
This allows you to save a pin and use the TrackPoint with a firmware that doesn’t support power-on-reset.
But if you have a pin to spare, you can connect the TrackPoint directly and configure this driver to perform the power-on-reset.
2.3.3. Press-To-Select (Clicking by pressing the TrackPoint)
TrackPoints have an interesting feature that allows you to click by pressing a little harder on the TrackPoint and this driver lets you enable it.
But, unfortunately, not all TrackPoints support this feature. My older Lenovo T430 TP supports it, but my newer, red T460S TP does not.
3. Acknowledgements
I also want to take a quick moment to thank some people.
Most of all to krikun98. Without your work this would not have been possible.
And to all the guys who have tirelessly helped me and answered my questions on discord. Most of all xudongz, joelspadin and of course petejohanson.
I look forward to your testing and feedback.
I really have to get #778 up to snuff, don't I. Great work here! I was planning to get to it eventually, but life got in the way and you beat me to it.
I really have to get #778 up to snuff, don't I. Great work here! I was planning to get to it eventually, but life got in the way and you beat me to it.
What do you think still needs to be done for #778? It looks pretty complete to me. Urob maintains a rebase of it on his fork and there are several people who are actively using it.
Maybe all of us can work on it together to get it to the finish line and merged.
Confirming it’s working on ZMK, using nrfmicro (nrf52840) and modified sofle.
I really have to get #778 up to snuff, don't I. Great work here! I was planning to get to it eventually, but life got in the way and you beat me to it.
What do you think still needs to be done for #778? It looks pretty complete to me. Urob maintains a rebase of it on his fork and there are several people who are actively using it.
Maybe all of us can work on it together to get it to the finish line and merged.
The parts relevant to this PR may look more or less complete, but mouse movement emulation needs a rewrite. I haven't looked at it in... a while, but from what I remember - it's using Zephyr's work queues to emulate ticks for the movement. They can get de-prioritized fairly easily, resulting in choppy movement over Bluetooth. I was planning to split up the PR and rewrite that part to utilize bare threads instead.
The parts relevant to this PR may look more or less complete, but mouse movement emulation needs a rewrite. I haven't looked at it in... a while, but from what I remember - it's using Zephyr's work queues to emulate ticks for the movement. They can get de-prioritized fairly easily, resulting in choppy movement over Bluetooth. I was planning to split up the PR and rewrite that part to utilize bare threads instead.
That makes sense and doesn’t sound like too much work. We should probably also check with Pete and the other maintainers to see what other requirements they might have.
I also briefly looked at integrating my mouse code with your mouse acceleration, but I didn’t want to intertwine the code too much.
So perhaps we could refactor that part a little too.
And maybe you could give me some guidance on how to send mouse events through the event system to the central half so that we can put the trackpoint on the peripheral side.
The parts relevant to this PR may look more or less complete, but mouse movement emulation needs a rewrite. I haven't looked at it in... a while, but from what I remember - it's using Zephyr's work queues to emulate ticks for the movement. They can get de-prioritized fairly easily, resulting in choppy movement over Bluetooth. I was planning to split up the PR and rewrite that part to utilize bare threads instead.
That makes sense and doesn’t sound like too much work. We should probably also check with Pete and the other maintainers to see what other requirements they might have.
I also briefly looked at integrating my mouse code with your mouse acceleration, but I didn’t want to intertwine the code too much.
So perhaps we could refactor that part a little too.
And maybe you could give me some guidance on how to send mouse events through the event system to the central half so that we can put the trackpoint on the peripheral side.
I knew the acceleration wasn't on in that demo video! I also wanted to make that part more extensible and easily configurable, maybe with on-the-fly profile tuning as in your trackpoint code. With mouse movement emulation having mouse movement keys on the peripheral is trivial - those were just new behaviours that plugged in to the matrix code. Here, it's a bit more difficult - I would imagine this would require somehow incorporating the trackpoint events into the central-peripheral communication. I never really investigated that part much, Pete or the people experimenting with wired split would surely know more.
And maybe you could give me some guidance on how to send mouse events through the event system to the central half so that we can put the trackpoint on the peripheral side.
It's worth considering sending mouse data the same way as the encoder stuff in #728.
It's worth considering sending mouse data the same way as the encoder stuff in #728.
Yeah that would be a good starting point. Although, I think one of the reasons the split side encoder stuff hasn’t been merged is that Pete didn’t think the communication was implemented the right way.
I think he is working right now on refactoring it. So perhaps it’s best to wait for those changes.
Wow, looks cool! Any updates? I will try to test it myself. I've got Cirque & Azteq touchpads if that helps anyone.
Wow, looks cool! Any updates? I will try to test it myself. I've got Cirque & Azteq touchpads if that helps anyone.
A couple of people have tried it and it seems to be working well enough for them. I am still in the process of designing my own keyboard with a trackpoint mount.
So I haven't had the opportunity to test it in real life and develop more new features.
I am not familiar with the circque & aztec touchpads, but if they use PS/2 I would love it if you tested them with this PR and gave feedback.
@infused-kim
As of now, I have tried soldering the pins and didn't get it working (Although, I am not sure whether I soldered the GND pin correctly or not). I will try again with another trackpad.
It'd be great if you could take a look the specs. The official site of it is cirque.com, though.
I've seen some post on Reddit / Discord of people having those working but in wired mode though.
@infused-kim
As of now, I have tried soldering the pins and didn't get it working (Although, I am not sure whether I soldered the GND pin correctly or not). I will try again with another trackpad.
It'd be great if you could take a look the specs. The official site of it is cirque.com, though.
I've seen some post on Reddit / Discord of people having those working but in wired mode though.
It seems like it uses i2c or spi
@infused-kim Yup! I followed the guide and removed the R1 resistor to enable i2c instead of SPI, but failed. Does the PR at its state imply that any I2C trackpad would be working by default?
P.S.
I got it. It wasn't supposed T_T
@infused-kim Yup! I followed the guide and removed the R1 resistor to enable i2c instead of SPI, but failed. Does the PR at its state imply that any I2C trackpad would be working by default?
P.S.
I got it. It wasn't supposed T_T
Yeah, unfortunately it’s for PS/2 pointing devices only. :)
@infused-kim Do you think it is going to be a lot of work to add the I2C support? A lot of users hope for that, as you've noticed on reddit :P
@infused-kim Do you think it is going to be a lot of work to add the I2C support? A lot of users hope for that, as you've noticed on reddit :P
I haven’t looked into it. So I can’t really say with any certainty.
What I can say with certainty though is that I am not interested in trackpads and that I won’t be working on that.
Sorry.
Hello everyone. I've been able to mount a T440 trackpoint to my keyboard running on a pair of nice!nano microcontrollers. Here's a picture of the finished build:
I have simply followed the steps in this PR and everything compiled, no issue.
So far, it seems to be useable enough for it to be my main way of moving the cursor :tada:
Following are the issues I've encountered for a day a half working with it:
- Firmware crashes every time I hold any key on the main side and moves the cursor for 6 seconds. This prevents using drag'n'drop or scrolling with btn3 pressed down. To recover from it, I have to reset the microcontroller. I had to move my mouse keys to the secondary side to avoid crashes, but it's actually more ergonomic that way, so I'm happy with that.
- I do encounter a few random crashes from time to time. Either it's the whole mcu or just the trackpoint that stops working. This is usually fixed by rebooting the mcu.
- I noticed that when I click a mouse button (any), the cursor moves slightly. It moves in the opposite direction from the last movement I made with the trackpoint. This prevents me from clicking on a draggable link, since it then moves the content.
- The trackpoint seems to drain my battery faster, but I have no good proof to show, apart from charging the keyboard overnight and wait for a full discharge. The battery usage doesn't show up any more in my computer (Manjaro Linux, Gnome, running on X11), but this has always been buggy...
CONFIG_ZMK_MOUSE_PS2_TP_TAP_TO_SELECT=ydoesn't seem to work at all for the T440, but I do sometimes get a random right click event happening. Though, this is very rare and might not be related.
I'd be nice if moving the cursor activated a layer dedicated to mouse clicks.
Again, beside the few issues I had, this is already very useable.
Hello everyone. I've been able to mount a T440 trackpoint to my keyboard running on a pair of nice!nano microcontrollers. Here's a picture of the finished build:
I have simply followed the steps in this PR and everything compiled, no issue.
So far, it seems to be useable enough for it to be my main way of moving the cursor 🎉
Following are the issues I've encountered for a day a half working with it:
- Firmware crashes every time I hold any key on the main side and moves the cursor for 6 seconds. This prevents using drag'n'drop or scrolling with btn3 pressed down. To recover from it, I have to reset the microcontroller. I had to move my mouse keys to the secondary side to avoid crashes, but it's actually more ergonomic that way, so I'm happy with that.
- I do encounter a few random crashes from time to time. Either it's the whole mcu or just the trackpoint that stops working. This is usually fixed by rebooting the mcu.
- I noticed that when I click a mouse button (any), the cursor moves slightly. It moves in the opposite direction from the last movement I made with the trackpoint. This prevents me from clicking on a draggable link, since it then moves the content.
- The trackpoint seems to drain my battery faster, but I have no good proof to show, apart from charging the keyboard overnight and wait for a full discharge. The battery usage doesn't show up any more in my computer (Manjaro Linux, Gnome, running on X11), but this has always been buggy...
CONFIG_ZMK_MOUSE_PS2_TP_TAP_TO_SELECT=ydoesn't seem to work at all for the T440, but I do sometimes get a random right click event happening. Though, this is very rare and might not be related.I'd be nice if moving the cursor activated a layer dedicated to mouse clicks.
Again, beside the few issues I had, this is already very useable.
Thank you for the detailed report. Very helpful.
I will look into all of these issues eventually. Especially the holding a key whole moving the mouse. If I can reproduce it, it should be easy to fix.
Unfortunately, at the moment I am busy with other stuff. So it might be awhile until I get to it.
Unfortunately, at the moment I am busy with other stuff. So it might be awhile until I get to it.
My C skills are a bit rusty but I'd be more than happy to contribute! I'll try and see how it goes as soon as I have some free time :)
Unfortunately, at the moment I am busy with other stuff. So it might be awhile until I get to it.
My C skills are a bit rusty but I'd be more than happy to contribute! I'll try and see how it goes as soon as I have some free time :)
Sweet. I would appreciate all the help I can get. Feel free to join #pointing-devices on the zmk discord and I’ll be happy to give you guidance.
Hey guys,
a few updates...
1. Rebase to latest zmk main
I have rebased this PR and the pr-testing branch onto the latest zmk main.
So, now you can enjoy your trackpoints as well as the latest zmk improvemements, such as split side encoders, timerless homerow mods, nice!view displays and much more.
2. My TP build
I have also finally finished building my own keyboard with a TP:
3. TP support improvements...
Since I now have a TP on my keyboard, I have a lot more motivation to work on the code. I am not ready to release it yet, but there is some exciting stuff coming up.
I am currently testing a new PS/2 driver that is using the nrf52's UART chip instead of GPIO interrupt. This should drastically reduce jumps and cutouts of the mouse movement.
And I am testing an auto-layer on mouse movement, which lets you turn your thumb cluster into mouse keys automatically when you are moving the trackpoint.
Both are working fairly well now, but not good enough for release.
If any of you are interested in testing it soon, let me know here or in the zmk discord #pointing-devices channel.
Wow~ this looks great. I was a long-time ThinkPad user and always wanted a track point on my custom keyboard. I've done that with QMK but unfortunately it can't work without wire. Very excited to see this. 😆
Hey guys,
a few updates... [clip] =
Is this still restricted to the central side only?
I am just playing with a cirque trackpad on i2c with QMK. It works really well! Much better than the pimoroni trackball. Hope this or something in the future will also work with i2c. I miss wireless on QMK. ;)
@infused-kim @tinyboxxx Just want to know where do you get the track point module? I have used one from T400 running QMK. The module uses an NXP chip which requires >4.5V power supply. So what types are capable under 3V power? I see @artggd mentioned T440, I will give it a try. I also notice ThinkPad uses a now module type starting from T440. Any other advices? Thank you.
@infused-kim @tinyboxxx Just want to know where do you get the track point module? I have used one from T400 running QMK. The module uses an NXP chip which requires >4.5V power supply. So what types are capable under 3V power? I see @artggd mentioned T440, I will give it a try. I also notice ThinkPad uses a now module type starting from T440. Any other advices? Thank you.
Most Lenovo trackpoints seem to work just fine on 3v. And I convinced an AliExpress store to sells the trackpoints without a keyboard.
I posted details on discord: https://discord.com/channels/719497620560543766/845285481888743434/1112605096262508645
@infused-kim @tinyboxxx Just want to know where do you get the track point module? I have used one from T400 running QMK. The module uses an NXP chip which requires >4.5V power supply. So what types are capable under 3V power? I see @artggd mentioned T440, I will give it a try. I also notice ThinkPad uses a now module type starting from T440. Any other advices? Thank you.
Most Lenovo trackpoints seem to work just fine on 3v. And I convinced an AliExpress store to sells the trackpoints without a keyboard.
I posted details on discord: https://discord.com/channels/719497620560543766/845285481888743434/1112605096262508645
Well, I found SK7102 - FlexPoint, which says it can run under 3.15V to 5.25V power. It looks very similar to the module I took from an old T400. It doesn't respond with ATMega32u4/8M/3.3V running QMK, but OK with 16M/5V. So it seems that QMK (or TMK underlying) is not fully compatible in its PS/2 driver, I guess.
I tested the trackpad. No matter how I modify the ZMK_MOUSE_PS2_SAMPLING_RATE transfer error bytes are a lot. Whenever I clicked the mouse button, the cursor moved randomly.
Hey guys,
I just finished the new UART based driver and a few other improvements.
What's new
UART Based PS/2 driver
The biggest change is that it's using the UART controller to receive data instead of GPIO interrupts.
PS/2 is a very old protocol and unfortunately not used in many devices anymore (apart from trackpoints). Therefore most chips don't include hardware to handle the communication with PS/2 devices (unlike, for example, SPI).
So, the original version of this driver used GPIO interrupts to bit-bang the PS/2 communication.
Unfortunately the nrf52840 chip used in nice!nanos wasn't quite fast enough to trigger all interrupts in time. This resulted in fairly frequent errors in transmission, which manifested as annoying cursor jumps or random clicks.
So I researched and played around with various protocols to try to find a way to process the PS/2 data through a hardware chip instead of software handled interrupts.
After many failed attempts, I was finally able to get it working using the nrf52 UART chip.
So, now you have the choice between a GPIO interrupt driver and the UART driver.
The UART driver uses UART to receive data and GPIO interrupt bit-banging to send data, because the nrf52 UART chip doesn't support the combination of start, parity and stop bits that PS/2 is using.
The performance is MUCH better with the new UART driver and it should be the one most people should use.
I have been using variations of this driver for a few weeks now and it has completely resolved all the issues I'v been having.
Zephyr 3.5 Input System
The second big change is that the PS/2 mouse driver has been updated for the new mouse PR that uses zephyr 3.5's input system.
We are getting much closer to getting official mouse support into zmk main. Pete recently added mouse clicks to main and he added a draft PR with a rewrite of the original mouse keys PR by krikun98 for the new input system in zephyr 3.5.
That rewrite made the mouse system much more stable and performant.
And this new update is takes advantage of these improvements.
But keep in mind that zephyr 3.5 is still highly experimental and there may be other bugs in the code-- potentially even in areas that are unrelated to this PR.
Auto Layer Switching on mouse movement
Another improvement I added is automatic layer switching on mouse movement. For example, you can activate a layer with mouse keys on your thumbs whenever you move the trackpoint.
And when you stop moving, it automatically deactivates.
You can also configure how long the mouse must be moved before the layer activates to prevent accidental activation during typing.
How to use it
If you never used this PR before...
You can use this commit as a guide on how to correctly enable mouse support.
The setup is fairly complex, but I added A LOT of comments that explain everything.
How to update from the old version
Almost everything has changed about the configuration. So it's best if you start fresh and follow the example commit above precisely.
Testing and feedback
The best way to report issues is here, but if you want to discuss it or have questions, it's best if you come to the zmk discord channel #pointing-devices.
I also created a thread in discord where I'll be posting more updates about the progress of this.
So make sure to subscribe if you want to stay up to date.
I tested the trackpad. No matter how I modify the ZMK_MOUSE_PS2_SAMPLING_RATE transfer error bytes are a lot. Whenever I clicked the mouse button, the cursor moved randomly. ! Thank you for the report.
@Fushilaofang can you please try the new version of the code (see the comment above this one for more info).