Power Management: long-running backups wake monitors from sleep
I have backintime run every hour via a cron job, but this means my monitors wake up every hour. This is a bit annoying if my laptop is where I am sleeping. Worse, for some reason the external display does not go back to sleep.
I'm running backintime with this command:
0 * * * * /usr/bin/nice -n19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime backup-job >/dev/null
I can confirm this behavior on Kubuntu 22.04. To reproduce:
- Open a sheel and call
sleep 10; backintime backup-job. - Before the 10 seconds have passed, turn off your monitors (e.g. by
xset dpms force off). - As soon as backintime starts, observe your monitors turning back on.
This is so strange!
Testing has determined that the inhibitSuspend function: https://github.com/bit-team/backintime/blob/46972353a49dee87017eba0b303c70501b355e25/common/tools.py#L1157 is not the cause of this.
To be continued …
I've been having a very hard time reliably reproducing this issue. Here's what I've found so far:
The screen will not wake up if the backup finishes quickly. Therefore, small test profiles with small amounts of data will not trigger this bug. It has to run for at least a second (gut feeling, not measured). This will be true for a profile with many files/folders, or with large amounts of data to be written.
In order to reliably reproduce the issue, I set up a profile such that it needed to copy several gigabytes into a new snapshot. This would wake up the monitors every time a backup runs (no matter if called by backintime backup on the console or backintime backup-job, which runs from cron). A convenient one-line command to observe the issue is:
xset dpms force off; sleep 1; backintime backup
If the backup takes sufficiently long to complete, you'll observe the monitors waking up.
For testing, I disabled both these functions:
https://github.com/bit-team/backintime/blob/ecf9312cdfc87f3417b30e883ee7cc89f0d285d1/common/tools.py#L1575
https://github.com/bit-team/backintime/blob/ecf9312cdfc87f3417b30e883ee7cc89f0d285d1/common/tools.py#L1615
by inserting return None as their first line.
This prevented the screens from waking up, so the issue is most likely with the "Inhibit suspend" functionality that is mediated through dbus, as defined here:
https://github.com/bit-team/backintime/blob/ecf9312cdfc87f3417b30e883ee7cc89f0d285d1/common/tools.py#L1552-L1573
I've tried to pin down whether it's inhibitSuspend or unInhibitSuspend that actually wakes up the monitor by inserting some debug messages and sleep() calls, but it's very tricky, because a) my monitor takes about 1–2s to wake up, and b) there seems to be some asynchronicity in the backup procedure that I haven't understood yet.
The solution should be to find the best-fitting arguments for this dbus call:
https://www.freedesktop.org/wiki/Software/systemd/inhibit/
We are currently using flags = INHIBIT_SUSPENDING | INHIBIT_IDLE
and I suspect INHIBIT_IDLE to wake up the display.
man gnome-session-inhibit says:
A typical use case is to prevent the session from going idle (and thus locking the screen) while a movie player is running.
@mankoff + @emtiu Could you please run my below scripts from within in /usr/share/backintime/common/ and test which one prevents your screen from going to sleep mode?
-
This is how BiT does it currently:
python3 -c "print('Inhibiting suspend...'); import tools; inhibitCookie = tools.inhibitSuspend(); input('Press a key to to re-enable suspend...'); inhibitCookie = tools.unInhibitSuspend(*inhibitCookie); print('un-inhibited suspend')" -
This is my proposed change (without
INHIBIT_IDLE):python3 -c "print('Inhibiting suspend...'); import tools; inhibitCookie = tools.inhibitSuspend(flags = tools.INHIBIT_SUSPENDING); input('Press a key to to re-enable suspend...'); inhibitCookie = tools.unInhibitSuspend(*inhibitCookie); print('un-inhibited suspend')"
You can check the active inhibit settings during the test with systemd-inhibit --list...
Note: Above test scripts are derived from:
https://github.com/bit-team/backintime/blob/9618d03520b902635e8b647a0ec6efa45bd8929e/common/tools.py#L1625-L1628
@mankoff + @emtiu Could you please run my below scripts from within in
/usr/share/backintime/common/and test which one prevents your screen from going to sleep mode?
In my case, it's not so easy. The bug is that when the monitors are already sleeping, a starting backintime job will wake them up (against my wish).
In my case, it's not so easy. The bug is that when the monitors are already sleeping, a starting backintime job will wake them up (against my wish).
Then you could test it with a sleep timer to start the script once the screen is blank and observe if it wakes up again:
sleep 600 && python3 -c "print('Inhibiting suspend...'); import tools; inhibitCookie = tools.inhibitSuspend(); input('Press a key to to re-enable suspend...'); inhibitCookie = tools.unInhibitSuspend(*inhibitCookie); print('un-inhibited suspend')"
sleep 600 && python3 -c "print('Inhibiting suspend...'); import tools; inhibitCookie = tools.inhibitSuspend(flags = tools.INHIBIT_SUSPENDING); input('Press a key to to re-enable suspend...'); inhibitCookie = tools.unInhibitSuspend(*inhibitCookie); print('un-inhibited suspend')"
Then you could test it with a
sleeptimer to start the script once the screen is blank and observe if it wakes up again:
Thanks, that helped to reproduce the issue once (for the first, unmodified code), but not realibly. I have no idea what's going on, but I'll think some more about it.