zmk
zmk copied to clipboard
zmkfirmware
Cannot connect to ZMK keyboard over Bluetooth on Pop_OS! 22.04 + Intel AX200 controller
Hi all,
For some reason (I believe after updating from Pop_OS! 20.10 to 22.04), I am no longer able to connect to either of my ZMK firmware keyboards over Bluetooth. I've collected all the logs I can think of below:
Bluetoothctl output:
Failed to pair: org.bluez.Error.AuthenticationFailed
dmesg output:
[ 9738.118236] Bluetooth: hci0: unexpected SMP command 0x0b from e1:ed:df:d8:a0:25
[ 9738.232020] Bluetooth: hci0: Opcode 0x 401 failed: -16
systemctl journal output:
src/device.c:load_gatt_db() No cache for E1:ED:DF:D8:A0:25
USB logging output:
[00:03:08.153,594] <dbg> zmk: zmk_endpoints_send_report: usage page 0x07
[00:03:19.176,818] <dbg> zmk: connected: Connected thread: 0x20004d90
[00:03:19.176,910] <dbg> zmk: connected: Connected 44:AF:28:56:A6:C0 (public)
[00:03:19.178,405] <dbg> zmk: update_advertising: advertising from 0 to 2
[00:03:19.213,256] <inf> bas: BAS Notifications enabled
[00:03:19.300,628] <err> zmk: Security failed: 44:AF:28:56:A6:C0 (public) level 1 err 4
[00:03:19.390,991] <inf> bas: BAS Notifications disabled
[00:03:19.393,676] <dbg> zmk: disconnected: Disconnected from 44:AF:28:56:A6:C0 (public) (reason 0x05)
[00:03:19.393,707] <dbg> zmk: update_advertising: advertising from 2 to 2
The Bluetooth controller is the built-in controller in the Intel AX200 chipset. I've tested all things I can think of, including manually installing the latest linux-firmware, reinstalling Pop_OS! from scratch, etc.
The connection does work properly on Windows and on another Linux laptop, plus my Android phone.
https://paste.sethforprivacy.com/?d908b857102a6e0d#6Csv3EbvFSz7HeSGTNCzdJupZkze4ZxAmoqVs3fVa5Wr
A friend of mine more familiar with ZMK suggested clearing all Bluetooth profiles on the keyboard (not just the one I was trying to use) and after doing so the issue is resolved.
If I can provide any additional logs to help track down this bug, please let me know -- I will leave this issue open for a bit in case that would be helpful!
Maybe the system upgrade wiped all the bonding informations, so the device can not be recognized anymore, and a sweep of Bluetooth profiles on keyboard may remove all the redundant bonds so they can be re-paired and re-bonded again.
Maybe the system upgrade wiped all the bonding informations, so the device can not be recognized anymore, and a sweep of Bluetooth profiles on keyboard may remove all the redundant bonds so they can be re-paired and re-bonded again.
The problem here is that the issue existed on two different ZMK-based keyboards, one of which was never connected before the upgrade to 22.04 as it didn't exist then.
I also cleared the first 4 profiles (the only ones I have ever tried to use on this keyboard) and that didn't resolve the issue, it was only resolved after clearing all profiles. Seems to be a bug of some sort that requires clearing all profiles from storage before you can properly connect in some scenarios.
I have the same recurring issue, running ZMK on two nice!nanos for my new split keyboard. Everytime I clear the profiles / reflash the firmware the issue goes away, but returns after one successful connected session using the keyboard over Bluetooth. Any Updates on this by anybody?
I have the same issue on nixos 23.11 Failed to pair: org.bluez.Error.AuthenticationFailed
I have done all the following : Removing the device, clearing all bluetooth profiles and flashing a settings reset firmware. After several attempts, I managed to pair the device in sudo mode.
But later I had to connect the device via usb to charge the battery, and after that the same error occur again when I try to pair the device.
Finally, pairing with bluetuith instead of bluetoothctl resolved the issue in my case. https://github.com/darkhz/bluetuith
Same issue on a glove80. Definitely a zmk strange behaviour I cannot understand (clear all only worked for me)
Same issue, when upgraded to ~~kernel 6.6.2~~. I tried clearing all profiles and reflashing, all I get is a connect and an immediate disconnect:
kernel: Bluetooth: hci0: unexpected SMP command 0x0b from eb:28:2c:3b:2e:ce
bluedevil-wizard[8227]: kf.bluezqt: PendingCall Error: "Authentication Canceled"
Edit: so the kernel is fine, the issue is related to the arch package linux-firmware-20231110.74158e7a-1. I have no idea who is to blame here.
happens to me again... this time clearing all profiles did not help... (kernel 6.6.3)
From the Bluetooth protocol point of view: Authentication failure is most likely to happen with a set of missing stored keys on central side, or to put it more plainly the host side (e.g. smartphones, PCs...). If the key is also/only missing on the peripheral side, central will initiate another pairing procedure to get a new set of keys so all the problems will gone.
From the keyboard view: There must be a upper limit for the count of stored keys, when the limit is breached, keys can get overlapped, discarded... and a similar restriction can consist in each slot, so a clearing of all profiles is always promising.
I first thought this was a kernel issue (works fine on linux 6.1, which I pinned for the time being) but I'm now also using a low energy pointing device which works perfectly fine, I'm starting to think that it's at least partially the fault of ZMK that this does not work
this is pretty serious, if ZMK doesn't support all linux kernels >6.4
Can you get a full btmon dump during a failed connection, so we can see what's afoot?
I will try to do so @petejohanson do you know someone who is running a zmk keyboard on a newer kernel where it's working fine? I have had this issue on all my nrf52840 boards for a while now... I have also reported an issue in the nixos repository, some other folks also reported problems with BLE on newer kernel and wanted to take it to the kernel bug report, the issue is that now I'm using another device that is BLE and works just fine so I started thinking it's not purely the kernels fault.
I will try to do so @petejohanson do you know someone who is running a zmk keyboard on a newer kernel where it's working fine? I have had this issue on all my nrf52840 boards for a while now... I have also reported an issue in the nixos repository, some other folks also reported problems with BLE on newer kernel and wanted to take it to the kernel bug report, the issue is that now I'm using another device that is BLE and works just fine so I started thinking it's not purely the kernels fault.
I will check my recently updated Fedora laptop to see the specific kernel version, but otherwise I've not been tracking anything at that granularity at this point. I will need to do so if we see a trend.
Can you also try my Zephyr 3.5 PR to see if anything improves?
I have not updated zmk for a while, I will try this first. If issues persist, I will put logs here anyway.
This is first running connnect, then after this fails pair.
le-chiff-ble-connect-6-6-zen.log
issues persist on firmware built with the latest 3.5 container
Just to be clear, is it also built using the 3.5 Zephyr branch of ZMK?
https://hub.docker.com/layers/zmkfirmware/zmk-build-arm/3.5-branch-20231114235846-3.5.0-0.16.3-6c450465d1dd-6870957618/images/sha256-33cffa362f3613197c0c37788415271f534b80f168bb56650ba1419d2f03d46d?context=explore
I used this one...
https://hub.docker.com/layers/zmkfirmware/zmk-build-arm/3.5-branch-20231114235846-3.5.0-0.16.3-6c450465d1dd-6870957618/images/sha256-33cffa362f3613197c0c37788415271f534b80f168bb56650ba1419d2f03d46d?context=explore
I used this one...
You'd need to actually use the ZMK branch from #1995 and west update. Just using the new container won't use a new Zephyr version under the hood.
my github workflow runs west update; does the container not have the 3.5 zephyr? why was it named that?
my github workflow runs west update; does the container not have the 3.5 zephyr? why was it named that?
It includes all the SDK and python dependencies needed to build something targeting Zephyr 3.5.
I have not updated zmk for a while, I will try this first. If issues persist, I will put logs here anyway.
This is first running connnect, then after this fails pair.
If this is logs from btmon then it's corrupted, perhaps.... Can we get btmon and dmesg output during the failure to connect?
btmon --read works fine for me, apparently it uses some binary format when doing --write
Another user reporting issues had a dmesg output indicating it was unhappy with the SMP security request packet... I do wonder if maybe using https://github.com/zmkfirmware/zmk/pull/2046 and enabling CONFIG_BT_GATT_AUTO_SEC_REQ=y will help.
btmon --read works fine for me, apparently it uses some binary format when doing --write
Ah, didn't know it had some binary format support. Looking now.
I also notice that the disconnects in both cases occur after the upgrade to 2M PHY... Perhaps related?
> HCI Event: LE Meta Event (0x3e) plen 6 #45 [hci0] 7.254650
LE PHY Update Complete (0x0c)
Status: Success (0x00)
Handle: 16 Address: D7:4E:99:F1:BE:98 (Static)
TX PHY: LE 2M (0x02)
RX PHY: LE 2M (0x02)
You also could try setting CONFIG_BT_CTLR_PHY_2M=n and see if that helps at all.