Suspend not working any more on qzed's 5.2.5 build
Hello!
I am using @qzed's latest 5.2.5 release 1 build, and I seem to have lost the ability to suspend my device; I close the lid, then pull it open a while later, and it's just as warm as when it started, instead of freezing cold, waking up a bit too quickly to have been suspended at all. I haven't changed anything kernel-related since I installed the build (does undocking and redocking for the first time since I installed the kernel count?), and I know the powerbutton isn't being pressed, as it's just sitting on my desk. In addition, how do I get the kernel logs related to whether it was suspended or not?
Any help on the matter would be greatly appreciated!
P.S. : WiFi module seems to disconnect when the lid is closed, and disappearing from the settings entirely at times, forcing me to reboot. Do I open a new issue for this? Bluetooth disconnects on lid close as well, though doesn't disappear from the settings.
Will something like the solution here work? https://askubuntu.com/a/1152359
UPDATE: It appears as though the Bluetooth module is constantly waking up the Surface. Anything I can do about that? Other than constantly turning off bluetooth before suspending?
Try this: https://github.com/jakeday/linux-surface/wiki/Frequently-Asked-Questions#my-device-wakes-up-automatically-from-suspendhibernate
EDIT: Though if the page you linked above didn't help, this wouldn't help either. The systemd service is mainly a way to make the changes to the /sys/.../power/wakeup survive reboot. The bluetooth wakeup should be at /sys/bus/pci/devices/0000:00:1d.3/power/wakeup
may a hint where to get the 525 kernel from ?
i d like to test on ubuntu 19 on surface laptop.
thanks ahead
@shadowrylander You could try to disable bluetooth via rfkill in the systemd sleep script and enable it after wakeup, if nothing else works. You could also have a look at /proc/acpi/wakeup, but I'm not sure if bluetooth is in there at all. The solution by @tmarkov should work though.
@alturismo You can find it here. I'll try to update the builds more frequently in the future.
@tmarkov They seem to be pretty much the same; I found out using the link I posted that it was the bluetooth module that was keeping the system awake, but not why.
@qzed As in my reply above, they seem to be the same help texts; instead, how do I edit the current scripts that control what happens when the lid is opened and closed? I'm thinking of using the rfkill to turn the module on and off, then reconnecting to my devices from the scripts as well.
@alturismo You can find it here. I'll try to update the builds more frequently in the future.
thanks, sadly same result with no keyboard and also cant replace the module like on 5.1.15 (wrong format) to get it working, so i wait ;)
@shadowrylander If so, why don't you just disable wakeup by bluetooth (you can make it survive reboot using systemd)? There shouldn't be any need to turn off the actual bluetooth.
@alturismo
thanks, sadly same result with no keyboard and also cant replace the module like on 5.1.15 (wrong format) to get it working, so i wait ;)
If it works with the module, it should also work with the kernel. For the module to work you will need to completely re-compile it for the kernel you're using (e.g. run make clean all each time you've installed a new kernel).
@shadowrylander as @tmarkov said: Find the bluetooth device (either in /proc/acpi/wakeup or under /sys/bus/...) and disable its wakeup capabilities.
The systemd sleep script (/lib/systemd/system-sleep/sleep) should really be your last option. Also this does trigger when you suspend, it doesn't have to do anything (directly) with the lid (other than lid causing suspend, which causes the script to be run).
@tmarkov I think probably because of this: https://bugs.launchpad.net/ubuntu/+source/bluez/+bug/1823076 And also because I forgot that was an option. I'll try that.
Also, to @qzed as well, could you tell me which of the entries are for the bluetooth module in /proc/acpi/wakeup, or if it should be clearly labelled bluetooth? I can't seem to find it. And where would it be under /sys/bus/... as well? acpi, for example?
Bluetooth is not in /proc/acpi/wakeup. On SB1 it is /sys/bus/pci/devices/0000:00:1d.3/power/wakeup but it may be different on other Surfaces. Honestly your confusion makes little sense to me: you said you found that bluetooth is responsible using this answer https://askubuntu.com/a/1152359, so you clearly must know where in /sys it is, which makes me think we're misunderstanding each other somehow.
@qzed thanks for the hint
@alturismo
thanks, sadly same result with no keyboard and also cant replace the module like on 5.1.15 (wrong format) to get it working, so i wait ;)
If it works with the module, it should also work with the kernel. For the module to work you will need to completely re-compile it for the kernel you're using (e.g. run
make clean alleach time you've installed a new kernel).
as its not persistent anyway i ll drop this test now with ubuntu 19.04 and go back to mint, personally best experience with surface laptop here.
https://github.com/jakeday/linux-surface/issues/563#issuecomment-530500845
@tmarkov Huh; I guess it works. I just realized I hadn't actually tested the solution after applying it. Sorry for the confusion; at this rate, no one's going to think my issues are legitimate! :sweat_smile: Thanks again for the help!
Quick note, however: the bluetooth wakeup file re-enables itself when I turn bluetooth back on, which means I would have to disable it again every time I wake the Surface up from suspend. Also, I just realized I never stated my device, which is a Surface Book 2 17". Sorry again!
That's true. So you can put the echo disabled > /sys/.../power/wakeup command in the pre) section of /lib/systemd/system-sleep/sleep script so it runs every time before you suspend.
Alternatively, you can setup a systemd service as explained here https://github.com/jakeday/linux-surface/wiki/Frequently-Asked-Questions#my-device-wakes-up-automatically-from-suspendhibernate
Two questions:
- I'm getting permission errors when trying to redirect echo to the wakeup file, even when using sudo, and
- I'm assuming the
systemdfile in the second example runs on every wakeup?
- Yes. Run
sudo suand thenecho disabled > /sus/.../power/wakeup. It will also run without permission errors from the system-sleep/sleep script or a systemd service. - I'm not 100% sure, but I think it would run on every boot. That said, the /power/wakeup values should survive suspend or hibernate, so every boot should be enough.
-
In addition to what @tmarkov wrote: If you want a one-liner:
echo disabled | sudo tee /sys/.../power/wakeuporsudo sh -c "echo disabled > /sys/.../power/wakeup"will do the trick. Permissions shouldn't be an issue in the systemd service file or sleep script, so no need to use sudo there. -
@tmarkov is correct,
WantedBy=multi-user.targetmeans it will run at boot. It could be that something like turning bluetooth off and on again changes the wakeup settings (as you've hinted), so you might need to put that into the systemd sleep script. Apart from that, those settings shouldn't change during suspend/resume.
@tmarkov and @qzed: So in the end, should I try the systemd script, then fall back to editing the system-sleep/sleep script if that doesn't work?
Sounds like a good plan
So I tried using the unit file in the systemd example, changing the lid sysfs accordingly, but it didn't seem to work after enabling it and rebooting, i.e. it didn't change the wakeup file contents to disabled. Then I tried changing the wakeup file for just the bluetooth module:
[Unit]
Description=Disable wakeup sources
[Service]
ExecStart=/bin/sh -c 'echo disabled > /sys/bus/usb/devices/1-6/power/wakeup'
Type=oneshot
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
but that didn't work either. I tried running the command under su (from sudo su), and it worked there. Is there anything else I'm missing? Or should I just switch to editing the system-sleep/sleep script?
Try systemctl status wakeup.service to make sure it's enabled. If not, run sudo systemctl enable wakeup.service - maybe sudo service enable wakeup doesn't work on newer systems?
If still doesn't work, use the system-sleep/sleep script. Keep in mind this repo overrides it, so if you run the setup.sh you'll need to edit it again.
I started it using sudo systemctl enable wakeup, then checked it's status after reboot; it said it ran just fine (though it doesn't seem to run successfully if Bluetooth wasn't on before restart). I may be forced to use the system-sleep/sleep script at this rate.
UPDATE: I think I'll have to switch to using rfkill to manage bluetooth from the system-sleep/sleep script; disabling the wakeup file didn't work.
UPDATE 2: It would appear as though the system-sleep/sleep script isn't working either; the bluetooth module seems to permanently block suspension. This is what I have right now in the script:
#!/bin/bash
case $1 in
pre)
# unload the modules before going to sleep
systemctl stop NetworkManager.service
modprobe -r intel_ipts
modprobe -r mei_me
modprobe -r mei
modprobe -r mwifiex_pcie;
modprobe -r mwifiex;
modprobe -r cfg80211;
####
rfkill block bluetooth
;;
post)
# need to cycle the modules on a resume and after the reset is called, so unload...
modprobe -r intel_ipts
modprobe -r mei_me
modprobe -r mei
modprobe -r mwifiex_pcie;
modprobe -r mwifiex;
modprobe -r cfg80211;
# and reload
modprobe -i intel_ipts
modprobe -i mei_me
modprobe -i mei
modprobe -i cfg80211;
modprobe -i mwifiex;
modprobe -i mwifiex_pcie;
echo 1 > /sys/bus/pci/rescan
systemctl restart NetworkManager.service
####
nmcli c up Primus-17FE
rfkill unblock bluetooth
echo "connect F0:13:C3:3B:B7:7F" | bluetoothctl
echo "connect E1:18:19:A6:A9:0B" | bluetoothctl
;;
esac
Do I need to add a wait after the rfkill block bluetooth command for it to take effect?
I'm pretty sure that /sys/bus/usb/devices/1-6/power/wakeup isn't bluetooth - after all, it's a usb device which bluetooth isn't (on SB1 it's the base) - and that's why rfkill-ing it doesn't help.
Can you confirm that the sleep script is executable?
Did you try disabling all sources but the ones you use rather than only 1-6?
The sleep scipt is indeed executable:
-rwxr-xr-x 1 root root 977 Sep 19 02:37 /lib/systemd/system-sleep/sleep
I searched up the idProduct and idVendor for 1-6 and found this, which is how I determined that it was the bluetooth module. The only other thing with an enabled wakeup script is my Logitech mouse; however, it seemed that my Surface would suspend just fine with that enabled and the bluetooth module disabled, so I thought that it wasn't the issue.
Here is what I get when I run a python script on the xonsh shell to get the devices and wakeup script status' on my shell:
-rw-r--r-- 1 root root 4096 Sep 19 09:58 /sys/bus/usb/devices/1-1.1.3/power/wakeup : enabled
-rw-r--r-- 1 root root 4096 Sep 19 09:58 /sys/bus/usb/devices/1-1.1/power/wakeup : disabled
-rw-r--r-- 1 root root 4096 Sep 19 09:58 /sys/bus/usb/devices/1-1.3/power/wakeup : disabled
-rw-r--r-- 1 root root 4096 Sep 19 09:58 /sys/bus/usb/devices/1-1/power/wakeup : disabled
-rw-r--r-- 1 root root 4096 Sep 19 09:58 /sys/bus/usb/devices/1-6/power/wakeup : disabled
-rw-r--r-- 1 root root 4096 Sep 19 09:58 /sys/bus/usb/devices/1-7/power/wakeup : disabled
-rw-r--r-- 1 root root 4096 Sep 19 09:58 /sys/bus/usb/devices/2-1.1/power/wakeup : disabled
-rw-r--r-- 1 root root 4096 Sep 19 09:58 /sys/bus/usb/devices/2-1/power/wakeup : disabled
-rw-r--r-- 1 root root 4096 Sep 19 09:58 /sys/bus/usb/devices/usb1/power/wakeup : disabled
-rw-r--r-- 1 root root 4096 Sep 19 09:58 /sys/bus/usb/devices/usb2/power/wakeup : disabled
Note that /sys/bus/usb/devices/1-6/power/wakeup becomes enabled when I turn bluetooth on; everything else stays the same. The script I used was this:
for i, j in zip($(ls -l /sys/bus/usb/devices/*/power/wakeup).split("\n"), $(cat /sys/bus/usb/devices/*/power/wakeup).split()):
print(i, ":", j)
Does your suspend work if you manually rfkill bluetooth before suspending?
Not all wakeup files are in /sys/bus/usb/devices/*/power/wakeup, so try this instead: find /sys -name wakeup -printf '%p ' -exec cat {} \;
Yes, rfkill blocking manually beforehand does indeed work. At this rate, is it possible that the bluetooth module activates something else as well that's preventing the Surface from suspending?
If it activated something else it would probably do it regardless of whether you rkdill block it manually or with a script... So it's worth looking at what happens differently between manual and script.
Do you remove modules manually before the rfkill? If not, you can try to move the rfkill in the sleep script before the modprobe -r. If that doesn't help, you can try to put a sleep 1 after the rfkill.
Does "removing modules manually" mean in the sleep script, or from the terminal? Because moving rfkill before the modprobe -r commands doesn't seem to be working either:
#!/bin/bash
case $1 in
pre)
# unload the modules before going to sleep
####
rfkill block bluetooth
####
systemctl stop NetworkManager.service
modprobe -r intel_ipts
modprobe -r mei_me
modprobe -r mei
modprobe -r mwifiex_pcie;
modprobe -r mwifiex;
modprobe -r cfg80211;
;;
post)
# need to cycle the modules on a resume and after the reset is called, so unload...
####
rfkill unblock bluetooth
echo "connect F0:13:C3:3B:B7:7F" | bluetoothctl
echo "connect E1:18:19:A6:A9:0B" | bluetoothctl
####
modprobe -r intel_ipts
modprobe -r mei_me
modprobe -r mei
modprobe -r mwifiex_pcie;
modprobe -r mwifiex;
modprobe -r cfg80211;
# and reload
modprobe -i intel_ipts
modprobe -i mei_me
modprobe -i mei
modprobe -i cfg80211;
modprobe -i mwifiex;
modprobe -i mwifiex_pcie;
echo 1 > /sys/bus/pci/rescan
systemctl restart NetworkManager.service
####
nmcli c up Primus-17FE
####
;;
esac
UPDATE: It would appear as though merely keeping the bluetooth module on isn't the issue; the connected devices seem to be preventing the Surface from suspending. So should I try disconnecting the devices from the sleep script on suspension, then reconnecting afterwards?
You could give that a try. I'm a bit confused why rfkill doesn't work, with that I'm a bit out of my depth here.
Well, I tried with rfkill, then sleeping for 5, as well as disconnecting from the devices then sleeping for 5, then rfkill and disconnecting then sleeping for 5, but that didn't work either; I know they're not working because my speakers keep reconnecting upon suspending. I'm kind of lost now.
Yes, rfkill blocking manually beforehand does indeed work.
Can you re-confirm that and give a step-by-step exactly what you do that works? Sorry if it's tedious asking you to re-confirm, but I have had situations when dealing with suspend problems when something seems to work and later it doesn't any more.
Oh, no problem at all! I just re-confirmed it. I ran rfkill block bluetooth and put the lid down, and the Surface didn't heat up at all, which usually happens when bluetooth devices are still connected while the Surface is being suspended.
For some reason, the post case in the sleep script seems to triggering as right after the pre case, though I'm not sure if this is why suspension isn't happening, or if this is happening because the device isn't suspending.
Does't heat up isn't reallu definitive. I usually play a song and see if it restarts on its own.
Also, perhaps there's something in the syslog? Can you give some logs from both cases?
My usual test case is connecting to my speakers before suspending, so when I do so, if the system doesn't suspend, it keeps connecting to my speakers, which keep playing the connected tone. The tone keeps playing repeatedly when turning off bluetooth in the sleep script pre section and turning it on again the post section, which shows that it keeps suspending, but waking up again right afterwards.
Also, where do I get the syslog for this again? I keep forgetting.
To get the syslog type journalctl. Please mark the specific sections where you suspend after running rfkill blobk bluetooth and it works; and when you suspend without and it doesn't work. Look for lines containing systemd[1]: Starting Suspend....
@shadowrylander I've just merged a PR (https://github.com/qzed/linux-surface/commit/da7bcd0efc8dfa40ea2f20ef40ec0094986d74f3) from @StollD using bluetoothctl to pretty much the same effect that we want to achieve here with rfkill. Can you, instead of rfkill try bluetoothctl power off and bluetoothctl power on?
@qzed @tmarkov Sorry about that; academics butting in. I'll try it; however, it seems as though the PR won't account for when devices are connected. Does this mean that when devices are connected, bluetooth will not power off?
@shadowrylander
it seems as though the PR won't account for when devices are connected. Does this mean that when devices are connected, bluetooth will not power off?
Yeah, that's right. I think in your case, you should comment out/remove the if clause and just run bluetoothctl power off unconditionally.
Ah, of course. That makes sense.Will do.
In addition, rfkill block bluetooth prevents bluetoothctl power on from working to turn it back on; this is normal, correct?
I think that should be the correct behavior.
My speakers still seem to be connecting repeatedly, meaning the the Surface is constantly waking up; this the current script:
#!/bin/bash
case $1 in
pre)
# unload the modules before going to sleep
####
bluetoothctl power off
####
modprobe -r intel_ipts
modprobe -r mei_me
modprobe -r mei
modprobe -r mwifiex_pcie;
modprobe -r mwifiex;
modprobe -r cfg80211;
;;
post)
# need to cycle the modules on a resume and after the reset is called, so unload...
####
bluetoothctl power on
bluetoothctl connect [MAC Address]
####
modprobe -r mei_me
modprobe -r mei
modprobe -r mwifiex_pcie;
modprobe -r mwifiex;
modprobe -r cfg80211;
# and reload
modprobe -i intel_ipts
modprobe -i mei_me
modprobe -i mei
modprobe -i cfg80211;
modprobe -i mwifiex;
modprobe -i mwifiex_pcie;
echo 1 > /sys/bus/pci/rescan
systemctl restart NetworkManager.service
# nmcli c up Primus-17FE
# sleep 5
;;
esac
may a question, how to check if this script is triggered on suspend/wakeup ?
i actually have the wifi dead or almost dead issue when i use suspend, sometimes i can recall wifi by using the cmd systemctl restart NetworkManager.service mostly not ... so i d like to check if the script is even running on suspend, wakeup ...
Edit, checked with simple echo`s and its running, so thats not the problem it seems.
But if you disable bluetooth before suspending it doesn't wake up? Can you upload a dmesg log of a suspend/resume cycle?
No; as far as I can tell, when devices are connected and I turn it off from the command line, it seems to keep waking up. When devices are not connected and I turn it off from the command line, it doesn't seem to wake up. And when I turn it off from Gnome Settings, it doesn't seem to wake up either.
Also, do you want the logs from when the Bluetooth power off command is in the sleep script, or done manually from the terminal?
For now just from the sleep script, maybe we can find something there.
@qzed @tmarkov Sorry; academics again. Also, I can't seem to make heads or tails of the logs; is there a way to remove anything unnecessary from it? Or should I just use pastebin or similar? Also, is there any sensitive information I should be worried about in it?
@qzed may a question as i reverted to 5.1.x kernel, i have the suspend / wifi issue with all newer kernels, as the rest is sorted (keyboard, touch) and here only wifi suspend is left, rather open a new issue or keep this one and i start to look for some debugs ?
@shadowrylander You can just pastebin the whole thing (or upload it as file here), better to have too much than too little. We'll probably only need the part involving suspend, so you could check what's all reported by journalctl before and just upload the parts that are being printed when you try to suspend and resume.
@alturismo Please open a new issue or have a look at the other wifi issues (I currently don't have an overview over that topic, so not sure if there's already one that fits your description).
@qzed Thanks! Here you go; sorry, there are sync program folder scans in between! logs.txt
@shadowrylander Looks like syncthing is the problem:
Oct 04 19:08:49 surface kernel: Freezing user space processes ...
Oct 04 19:08:49 surface kernel: Freezing of tasks failed after 20.003 seconds (1 tasks refusing to freeze, wq_busy=0):
Oct 04 19:08:49 surface kernel: syncthing D 0 3281 1 0x00004324
Oct 04 19:08:49 surface kernel: Call Trace:
Oct 04 19:08:49 surface kernel: __schedule+0x2bb/0x660
Oct 04 19:08:49 surface kernel: schedule+0x2d/0x90
Oct 04 19:08:49 surface kernel: request_wait_answer+0xa3/0x210
Oct 04 19:08:49 surface kernel: ? wait_woken+0x80/0x80
Oct 04 19:08:49 surface kernel: __fuse_request_send+0x69/0x90
Oct 04 19:08:49 surface kernel: fuse_request_send+0x29/0x30
Oct 04 19:08:49 surface kernel: fuse_simple_request+0xdd/0x1a0
Oct 04 19:08:49 surface kernel: fuse_do_getattr+0xe3/0x300
Oct 04 19:08:49 surface kernel: fuse_perm_getattr+0x26/0x30
Oct 04 19:08:49 surface kernel: fuse_permission+0x100/0x150
Oct 04 19:08:49 surface kernel: inode_permission+0xd8/0x1a0
Oct 04 19:08:49 surface kernel: link_path_walk.part.42+0x29e/0x540
Oct 04 19:08:49 surface kernel: path_lookupat.isra.45+0x93/0x220
Oct 04 19:08:49 surface kernel: filename_lookup.part.61+0xa0/0x170
Oct 04 19:08:49 surface kernel: ? __check_object_size+0x166/0x192
Oct 04 19:08:49 surface kernel: ? strncpy_from_user+0x57/0x1b0
Oct 04 19:08:49 surface kernel: user_path_at_empty+0x3e/0x50
Oct 04 19:08:49 surface kernel: vfs_statx+0x76/0xe0
Oct 04 19:08:49 surface kernel: __do_sys_newlstat+0x3d/0x70
Oct 04 19:08:49 surface kernel: ? __secure_computing+0x3e/0xd0
Oct 04 19:08:49 surface kernel: ? syscall_trace_enter+0x196/0x2b0
Oct 04 19:08:49 surface kernel: __x64_sys_newlstat+0x16/0x20
Oct 04 19:08:49 surface kernel: do_syscall_64+0x5a/0x140
Oct 04 19:08:49 surface kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Oct 04 19:08:49 surface kernel: RIP: 0033:0x4abb90
Oct 04 19:08:49 surface kernel: Code: Bad RIP value.
Oct 04 19:08:49 surface kernel: RSP: 002b:000000c4227bb8a0 EFLAGS: 00000202 ORIG_RAX: 0000000000000006
Oct 04 19:08:49 surface kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00000000004abb90
Oct 04 19:08:49 surface kernel: RDX: 0000000000000000 RSI: 000000c422020928 RDI: 000000c422041a10
Oct 04 19:08:49 surface kernel: RBP: 000000c4227bb908 R08: 0000000000000000 R09: 0000000000000000
Oct 04 19:08:49 surface kernel: R10: 0000000000000000 R11: 0000000000000202 R12: ffffffffffffffff
Oct 04 19:08:49 surface kernel: R13: 000000000000008c R14: 000000000000008b R15: 00000000000000aa
Oct 04 19:08:49 surface kernel: OOM killer enabled.
Oct 04 19:08:49 surface kernel: Restarting tasks ... done.
Oct 04 19:08:49 surface systemd-sleep[3209]: Failed to suspend system. System resumed again: Device or resource busy
In the logs, it looks like the device is trying to suspend multiple times, but each time is blocked by syncthing.
I thought so too at first, but then remembered that, for some reason, it only stops suspending when bluetooth is active, and as far as I know, syncthing and bluetooth do not interact with each other. Syncthing works regardless of whether bluetooth on on or not.
So I have no idea what's happening.
UPDATE: Actually, now it's starting to happen even with Bluetooth off; I think trying to shut off Syncthing from the sleep script might work. I'll test it with Bluetooth on to see if it still suspends, as that may be the real culprit here.
I am not able to suspend either, SB running 19.04 however I don't have syncthing installed. Not sure if this is a generic issue or one related to syncthing ?
$ uname -a Linux esd01 5.3.10-surface #1 SMP Sun Nov 10 20:33:54 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Apparently upgrading to Ubuntu 19.10 fixed the issue; not entirely sure why. Should I mark this issue as solved, then?