SSH-Agent error on Ubuntu : Could not add card, agent refused operation

[xyg6]

Hello,

I'm on Ubuntu 20, up to date, and I've followed the steps to make this tool with success and I could generate certificates on my Yubikey 5 via command line.

I've looked at the issues here on Github stating the same error I get when I want to use ssh-agent, but without a solution.

When I launch the command ssh-add -s /usr/local/lib/libykcs11.so it prompts me for the PIV password. I enter it and the key doesn't blink as usually when an action triggers it, and the prompt errors

❯ ssh-add -s /usr/local/lib/libykcs11.so
Enter passphrase for PKCS#11: 
Could not add card "/usr/local/lib/libykcs11.so": agent refused operation

I also tried installing piv-tool via apt, but it still doesn't work

sudo apt install yubico-piv-tool

How to debug or fix this issue ?

[a-dma]

Hi,

the easiest way to get more information would probably be to manually start ssh-agent in debug mode with ssh-agent -d in a separate terminal. Additionally ssh-add also accepts one or more -v options to increase verbosity.

You can also enable debug output from ykcs11 itself by setting the environment variable YKCS11_DBG.

I also tried installing piv-tool via apt, but it still doesn't work

how else have you installed it? Manually built? The path that you mentioned above looks correct but the command that you pasted installs just yubico-piv-tool, if you want to install ykcs11 you need to explicitly install the ykcs11 package.

Finally, ssh-agent has a -P option for allowed PKCS#11 providers, if you're placing the .so file somewhere other than one of the default paths (/usr/lib/*,/usr/local/lib/*) then you must provide that path when starting ssh-agent.

[darkobas2]

my finding is there is a problem with PIV interface somehow doesnt work when you plug in the key. workaround is to open the yubikey manager gui. got to interfaces, disable PIV, save config, enable PIV and save configuration... now if you go to applications/piv you should be able to click PIV without error and also use ssh-add without issue

[qpernil]

Would you be able to start the ssh-agent in debug mode as described above, with the YKCS11_DBG env variable ? The reason this would be interesting is that I've seen on an Ubuntu laptop of my own that something to do with smartcard login on the machine seems to grab my YubiKey exclusively as soon as I insert it. Reconfiguring the USB interfaces like you did causes a usb re-enumeration which might mean that whatever it is that was grabbing the device is no longer doing so at that point.

Another thing you could try, to isolate if this is something to do with PIV or PCSC vs ssh-agent, would be to just run yubico-piv-tool -astatus -v3 before and after trying your workaround, and post the output here.

[darkobas2]

@qpernil Hi! thanx for interest. Indeed, just unplugging and re-plugging the stick makes it work. So its an issue that during boot its somehow not initialized correctly or completely. yubico-piv-tool -astatus -v3 didnt find any yubikeys in this case.

[qpernil]

The issue that happens on my machine is that the login process takes exclusive access to the PC-SC reader, preventing yubico-piv-tool from accessing it. If you run the ssh-agent, or yubico-piv-tool, with debugging on we could probably deduce what is happening. Unplugging and re-plugging forcibly invalidates the handle the login process has, and it doesn't seem to retry while you're already logged in.

[darkobas2]

Hi

using dbg on piv or ssh doesnt seem to do much as the problem seems to go deeper.

$ yubico-piv-tool -astatus -v3
SCardListReaders failed, rc=8010002e
No usable reader found matching 'Yubikey'.
Failed to connect to yubikey.
Try removing and reconnecting the device.d

all on yubi since boot:

feb 25 19:15:45 darkfund kernel: usb 1-4: Product: YubiKey OTP+FIDO+CCID
feb 25 19:15:45 darkfund kernel: usb 1-4: Manufacturer: Yubico
feb 25 19:15:45 darkfund kernel: input: Yubico YubiKey OTP+FIDO+CCID as /devices/pci0000:00/0000:00:08.1/0000:04:00.3/usb1/1-4/1-4:1.0/0003:1050:0407.0002/input/input10
feb 25 19:15:45 darkfund kernel: hid-generic 0003:1050:0407.0002: input,hidraw1: USB HID v1.10 Keyboard [Yubico YubiKey OTP+FIDO+CCID] on usb-0000:04:00.3-4/input0
feb 25 19:15:45 darkfund kernel: hid-generic 0003:1050:0407.0003: hiddev0,hidraw2: USB HID v1.10 Device [Yubico YubiKey OTP+FIDO+CCID] on usb-0000:04:00.3-4/input1
feb 25 19:15:45 darkfund systemd[1]: Mounting Mount unit for yubioath-desktop, revision 13...
feb 25 19:15:46 darkfund systemd-udevd[841]: 3-3:1.1: Process '/usr/lib/snapd/snap-device-helper bind snap_yubioath-desktop_pcscd /devices/pci0000:00/0000:00:08.1/0000:04:00.4/usb3/3-3/3-3:1.1 0:0' failed with exit code 1.
feb 25 19:15:46 darkfund systemd-udevd[841]: 3-3:1.1: Process '/usr/lib/snapd/snap-device-helper bind snap_yubioath-desktop_yubioath-desktop /devices/pci0000:00/0000:00:08.1/0000:04:00.4/usb3/3-3/3-3:1.1 0:0' failed with exit code 1.
feb 25 19:15:46 darkfund systemd-udevd[818]: 3-3:1.0: Process '/usr/lib/snapd/snap-device-helper bind snap_yubioath-desktop_pcscd /devices/pci0000:00/0000:00:08.1/0000:04:00.4/usb3/3-3/3-3:1.0 0:0' failed with exit code 1.
feb 25 19:15:46 darkfund systemd-udevd[819]: 3-1:1.1: Process '/usr/lib/snapd/snap-device-helper bind snap_yubioath-desktop_pcscd /devices/pci0000:00/0000:00:08.1/0000:04:00.4/usb3/3-1/3-1:1.1 0:0' failed with exit code 1.
feb 25 19:15:46 darkfund systemd-udevd[819]: 3-1:1.1: Process '/usr/lib/snapd/snap-device-helper bind snap_yubioath-desktop_yubioath-desktop /devices/pci0000:00/0000:00:08.1/0000:04:00.4/usb3/3-1/3-1:1.1 0:0' failed with exit code 1.
feb 25 19:15:46 darkfund systemd-udevd[818]: 3-3:1.0: Process '/usr/lib/snapd/snap-device-helper bind snap_yubioath-desktop_yubioath-desktop /devices/pci0000:00/0000:00:08.1/0000:04:00.4/usb3/3-3/3-3:1.0 0:0' failed with exit code 1.
feb 25 19:15:46 darkfund systemd-udevd[857]: 3-1:1.0: Process '/usr/lib/snapd/snap-device-helper bind snap_yubioath-desktop_pcscd /devices/pci0000:00/0000:00:08.1/0000:04:00.4/usb3/3-1/3-1:1.0 0:0' failed with exit code 1.
feb 25 19:15:46 darkfund systemd-udevd[857]: 3-1:1.0: Process '/usr/lib/snapd/snap-device-helper bind snap_yubioath-desktop_yubioath-desktop /devices/pci0000:00/0000:00:08.1/0000:04:00.4/usb3/3-1/3-1:1.0 0:0' failed with exit code 1.
feb 25 19:15:46 darkfund systemd[1]: Mounted Mount unit for yubioath-desktop, revision 13.
feb 25 19:15:48 darkfund audit[1273]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap-update-ns.yubioath-desktop" pid=1273 comm="apparmor_parser"
feb 25 19:15:48 darkfund audit[1301]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap.yubioath-desktop.pcscd" pid=1301 comm="apparmor_parser"
feb 25 19:15:48 darkfund audit[1300]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap.yubioath-desktop.hook.configure" pid=1300 comm="apparmor_parser"
feb 25 19:15:48 darkfund audit[1302]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="snap.yubioath-desktop.yubioath-desktop" pid=1302 comm="apparmor_parser"
feb 25 19:15:48 darkfund systemd[1]: Started Service for snap application yubioath-desktop.pcscd.
feb 25 19:15:49 darkfund yubioath-desktop.pcscd[1440]: 00000000 readerfactory.c:1106:RFInitializeReader() Open Port 0x200000 Failed (usb:1050/0407:libudev:0:/dev/bus/usb/001/002)
feb 25 19:15:49 darkfund yubioath-desktop.pcscd[1440]: 00000072 readerfactory.c:376:RFAddReader() Yubico YubiKey OTP+FIDO+CCID init failed.
feb 25 19:15:49 darkfund yubioath-desktop.pcscd[1440]: 00006285 readerfactory.c:1106:RFInitializeReader() Open Port 0x200000 Failed (usb:1050/0407:libudev:1:/dev/bus/usb/001/002)
feb 25 19:15:49 darkfund yubioath-desktop.pcscd[1440]: 00000014 readerfactory.c:376:RFAddReader() Yubico YubiKey OTP+FIDO+CCID init failed.
feb 25 19:16:37 darkfund /usr/lib/gdm3/gdm-x-session[2591]: (II) config/udev: Adding input device Yubico YubiKey OTP+FIDO+CCID (/dev/input/event7)
feb 25 19:16:37 darkfund /usr/lib/gdm3/gdm-x-session[2591]: (**) Yubico YubiKey OTP+FIDO+CCID: Applying InputClass "libinput keyboard catchall"
feb 25 19:16:37 darkfund /usr/lib/gdm3/gdm-x-session[2591]: (II) Using input driver 'libinput' for 'Yubico YubiKey OTP+FIDO+CCID'
feb 25 19:16:37 darkfund /usr/lib/gdm3/gdm-x-session[2591]: (**) Yubico YubiKey OTP+FIDO+CCID: always reports core events
feb 25 19:16:37 darkfund /usr/lib/gdm3/gdm-x-session[2591]: (II) event7  - Yubico YubiKey OTP+FIDO+CCID: is tagged by udev as: Keyboard
feb 25 19:16:37 darkfund /usr/lib/gdm3/gdm-x-session[2591]: (II) event7  - Yubico YubiKey OTP+FIDO+CCID: device is a keyboard
feb 25 19:16:37 darkfund /usr/lib/gdm3/gdm-x-session[2591]: (II) event7  - Yubico YubiKey OTP+FIDO+CCID: device removed
feb 25 19:16:37 darkfund /usr/lib/gdm3/gdm-x-session[2591]: (II) XINPUT: Adding extended input device "Yubico YubiKey OTP+FIDO+CCID" (type: KEYBOARD, id 9)

[darkobas2]

@qpernil so the problem is caused by yubioauth-desktop

snap remove yubioath-desktop

and now it seems to work without replug (at least on first try hope it wasnt some weird luck). I dont even need the yubioauth app

[qpernil]

OK, thx for investigating this issue !

[mouse07410]

This probably has to do with de-selection of PIV applet on the token? And lack of "smarts" in either SSH or YKCS11 to ensure the applet wasn't deselected from underneath it and re-select it if that's the case?

[qpernil]

It's more fundamental, the systemwide pcsc dameon seems to be unable to access the usb device. Looking into what yubioath-desktop is doing, will report back here soon.

[darkobas2]

@mouse07410 i wouldn't know that... All i can add is that not only ssh or yubico-piv-tool dont work but also the yubi-manager app doesnt work with PIV until you disable piv and reenable it

[qpernil]

Yes, that makes sense given the above.

[qpernil]

@darkobas2 would you mind letting me know how you installed pcscd ? Via apt or another snap ? Or other means, like building it yourself locally ? As far as I know it's not installed by default on Ubuntu20. Please let me know if I'm wrong on that one.

[darkobas2]

@qpernil installed with apt

Package: pcscd
Version: 1.8.26-3
Priority: extra
Section: universe/misc
Source: pcsc-lite
Origin: Ubuntu
Maintainer: Ubuntu Developers <[email protected]>
Original-Maintainer: Ludovic Rousseau <[email protected]>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 179 kB
Pre-Depends: init-system-helpers (>= 1.54~)
Depends: libccid (>= 1.4.1~) | pcsc-ifd-handler, libc6 (>= 2.15), libsystemd0, libudev1 (>= 183), lsb-base (>= 3.0-6), libpcsclite1 (= 1.8.26-3)
Suggests: systemd
Homepage: https://pcsclite.apdu.fr/
Download-Size: 58,1 kB
APT-Manual-Installed: no
APT-Sources: http://si.archive.ubuntu.com/ubuntu focal/universe amd64 Packages

[qpernil]

Please see the text for https://snapcraft.io/yubioath-desktop

I contains it's own bundled pcscd, and is incompatible with the systemwide pcscd (running at the same time)

So to be able to run yubico-piv-tool:

sudo snap stop yubioath-desktop.pcscd sudo systemctl start pcscd sudo systemctl start pcscd.socket

To go back to running yubioath-desktop:

sudo systemctl stop pcscd sudo systemctl stop pcscd.socket sudo snap restart yubioath-desktop.pcscd

It is possible future versions of yubioath-desktop will stop using snap, but this is the current situation.

[qpernil]

8010002e means pcscd can't even list the reader, which normally wouldn't happen even if some app had it opened exclusively.

[qpernil]

Does the above solve this issuer for the original poster ?

[qpernil]

Closing this issue now as it seems to be a problem unrelated to yubico-piv-tool. Please feel free to open new issues as needed.

[darkobas2]

Yes, we figured the "what" and thats what matter s and it helped me. Thank you for your time.