client
client copied to clipboard
keybase
kbfs not allowing laptop to suspend
System
- KUbuntu 18.04
- 4.15.0-23-generic kernel
- keybase version 2.1.0-20180615162035+8625b772b1
What happens
With keybase running, if I access kbfs (either through the terminal or file explorer) and go about my day, when I attempt to suspend my laptop, the system locks, then kind of hangs, then wakes back up. Exploring the syslog, I found that something to do with fuse was causing it. See the log below.
I then killed all of keybase doing the following:
$ killall Keybase
$ fusermount -uz /keybase
fusermount: entry for /keybase not found in /etc/mtab
$ killall keybase
$ killall kbfsfuse
I'm able to suspend my PC now just fine. It's essentially every time I access kbfs that this occurs.
Jun 19 16:50:53 devel systemd[1]: Starting Suspend...
Jun 19 16:50:53 devel wpa_supplicant[1012]: nl80211: deinit ifname=wlp58s0 disabled_11b_rates=0
Jun 19 16:50:53 devel systemd-sleep[31482]: Suspending system...
Jun 19 16:50:53 devel kernel: [28295.483311] PM: suspend entry (deep)
Jun 19 16:50:53 devel kernel: [28295.483313] PM: Syncing filesystems ... done.
Jun 19 16:51:13 devel kernel: [28295.494617] Freezing user space processes ...
Jun 19 16:51:13 devel kernel: [28315.498453] Freezing of tasks failed after 20.003 seconds (1 tasks refusing to freeze, wq_busy=0):
Jun 19 16:51:13 devel kernel: [28315.498543] ksysguardd D 0 2223 1869 0x00000004
Jun 19 16:51:13 devel kernel: [28315.498549] Call Trace:
Jun 19 16:51:13 devel kernel: [28315.498566] __schedule+0x297/0x8b0
Jun 19 16:51:13 devel kernel: [28315.498574] schedule+0x2c/0x80
Jun 19 16:51:13 devel kernel: [28315.498581] request_wait_answer+0xa3/0x210
Jun 19 16:51:13 devel kernel: [28315.498587] ? wait_woken+0x80/0x80
Jun 19 16:51:13 devel kernel: [28315.498592] __fuse_request_send+0x86/0x90
Jun 19 16:51:13 devel kernel: [28315.498597] fuse_request_send+0x27/0x30
Jun 19 16:51:13 devel kernel: [28315.498601] fuse_simple_request+0xcf/0x1a0
Jun 19 16:51:13 devel kernel: [28315.498608] fuse_statfs+0xe1/0x150
Jun 19 16:51:13 devel kernel: [28315.498617] statfs_by_dentry+0x6d/0x90
Jun 19 16:51:13 devel kernel: [28315.498622] vfs_statfs+0x1b/0xc0
Jun 19 16:51:13 devel kernel: [28315.498627] user_statfs+0x5a/0xa0
Jun 19 16:51:13 devel kernel: [28315.498633] SYSC_statfs+0x27/0x60
Jun 19 16:51:13 devel kernel: [28315.498640] SyS_statfs+0xe/0x10
Jun 19 16:51:13 devel kernel: [28315.498647] do_syscall_64+0x73/0x130
Jun 19 16:51:13 devel kernel: [28315.498652] entry_SYSCALL_64_after_hwframe+0x3d/0xa2
Jun 19 16:51:13 devel kernel: [28315.498658] RIP: 0033:0x7fcc31773977
Jun 19 16:51:13 devel kernel: [28315.498661] RSP: 002b:00007ffcc0d6b628 EFLAGS: 00000246 ORIG_RAX: 0000000000000089
Jun 19 16:51:13 devel kernel: [28315.498666] RAX: ffffffffffffffda RBX: 000055da4fca526a RCX: 00007fcc31773977
Jun 19 16:51:13 devel kernel: [28315.498668] RDX: 000055da4fcf9a80 RSI: 00007ffcc0d6b630 RDI: 000055da4fca526a
Jun 19 16:51:13 devel kernel: [28315.498670] RBP: 00007ffcc0d6b630 R08: 000055da4fcf9810 R09: 0000000000000000
Jun 19 16:51:13 devel kernel: [28315.498673] R10: 000055da4fca2010 R11: 0000000000000246 R12: 000055da4fcf9a10
Jun 19 16:51:13 devel kernel: [28315.498675] R13: 000055da4f423b3a R14: 0000000100002600 R15: 000055da4fcf9810
Jun 19 16:51:13 devel kernel: [28315.498758] OOM killer enabled.
Hmm, that's frustrating. I see a matching kernel bug report for FUSE: https://bugzilla.kernel.org/show_bug.cgi?id=198879
But I've never experienced myself on this on windows.
For the record, unless you've changed the default Keybase config, /keybase is redirector mount that's run by root, and there's another mountpoint running at something like /run/user/1000/keybase/kbfs. But killall kbfsfuse should take care of that one.
If you have open terminals or explorer windows under KBFS, does it help to back out of them first before you try to suspend?
@strib
If you have open terminals or explorer windows under KBFS, does it help to back out of them first before you try to suspend?
It doesn't help, but I can't seem to get the issue to trigger if I've:
- Never opened kbfs through terminal/file explorer, but keybase is running
- Only viewed the files through the keybase application
I've currently created on-suspend and on-wake scripts for my system that basically kill kbfs when suspending, and start it back up on wake. I'm not sure if this could be a workaround for the keybase app on Linux, though.
@nunofernandes You can use my script to help. I haven't experienced the issue since I wrote this:
https://gist.github.com/psidhu/a65ab9632bd6c6d400ea3fe25e959fa0
Near the end, change ${TODO_CHANGE_THIS_VAR_TO_YOUR_ACTUAL_USER_NAME_MANUALLY} to your username, e.g. nunofernandes. Place the script into /lib/systemd/system-sleep/keybase-suspend and mark it executable.
I've only tested this on Ubuntu 18.04.
What I did on systemd system (Debian unstable):
Created /etc/systemd/system/[email protected] file based on nice recommendations in Arch Wiki "suspend/resume" and "Combined Suspend/resume service file" chapters.
[Unit]
Description=Stop/Restart kbfs before suspend (bug #12458)
Before=sleep.target
StopWhenUnneeded=yes
[Install]
WantedBy=sleep.target
[Service]
User=%I
Type=oneshot
Environment=XDG_RUNTIME_DIR=/run/user/%I
ExecStart=-/bin/systemctl --user stop kbfs.service
ExecStop=-/bin/systemctl --user start kbfs.service
RemainAfterExit=yes
sudo systemctl daemon-reload
Then I enabled it for my desktop user id (1000 on a typical system, find with id -u):
sudo systemctl enable [email protected]
Unfortunately, in order for this to work, kbfs.service needs to support stop command. Either https://github.com/keybase/kbfs/pull/1877 should be merged and released first or you can create a systemd drop-in in the meantime:
/etc/systemd/user/kbfs.service.d/10-stop.conf
[Service]
ExecStop=-/bin/sh -c 'fusermount -uz "$(keybase config get -d -b mountdir)"'
systemctl --user daemon-reload
This should unmount kbfs fuse filesystem before suspend (stop kbfs systemd-user service) and remount it back (start kbfs systemd-user service) once machine wakes up, hence avoiding hitting this kernel bug. The problem with this solution is that it is not really suitable for multiuser systems as keybase-suspend@ system scope systemd service must be explicitly enabled for all users. But it should be fine for typical desktop/laptop usage.
It seems that the kbfs FUSE filesystem not returning a result from statfs() promptly is also indirectly preventing Flatpak apps from starting while kbfs is mounted in a top-level directory like /keybase (flatpak/flatpak#5496).
FUSE filesystems should really be prepared to return a result from statfs() at any time, even if that means reading a possibly out of date answer from a cache. Other user-space components like Flatpak cannot reliably recognise FUSE filesystems without that working, because statfs() is precisely how we find out that it's a FUSE filesystem in the first place.
If this is causing a kernel hang during suspend, then I think that's additional evidence that whatever kbfs is doing in its statfs() implementation, it's more complicated than what the kernel expects to be valid to do there.
Can confirm issue reproduces on Ubuntu 22.04 LTS with Keybase and flatpak installed, and restart keybase-redirector and kbfs service workarounds the issue temporarily as mentioned in https://github.com/flatpak/flatpak/issues/5496#issuecomment-1712653275
# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"
# uname -a
Linux roach 6.1.0-1027-oem #27-Ubuntu SMP PREEMPT_DYNAMIC Mon Nov 20 15:08:14 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
https://github.com/flatpak/flatpak/issues/5496#issuecomment-1869387581
