fohrloop
Using system python in a subprocess to enter wakepy Modes
Background: This idea came with and experiment for the gtk_application_inhibit() based Method (see PR 407). I used a subprocess which ran system python and utilized a "inhibitor.py" module, which utilized the PyGObject (import name gi) package.
Idea: Would it be possible to do like this:
- Start subprocess with system python
- Use the libraries from system python, like jeepney, dbus-python and wakepy from the venv.
While it feels a bit unorthodox, is this technically possible? Are there some culprits or downsides? This way wakepy could be completely (python) dependency free, meaning that installing wakepy would not require installing any python packages on modern python versions (3.10+). That would close for example https://github.com/fohrloop/wakepy/issues/65.
Potential problems:
One potential problem is to make sure the methodology works with bundlers like PyInstaller, but that's a problem for later.
I created a PoC for this and it seems to be possible to do this (nice! 🎉)
One consequence of doing this is that all the Methods that will support also system python, will be duplicated with a separate name. So wakepy could have for example following methods:
caffeinate
org.freedesktop.PowerManagement
org.freedesktop.PowerManagement [SYS]
org.freedesktop.ScreenSaver
org.freedesktop.ScreenSaver [SYS]
org.gnome.SessionManager
org.gnome.SessionManager [SYS]
gtk_application_inhibit
gtk_application_inhibit [SYS]
SetThreadExecutionState
When the Methods using system python are under separate names:
- it will be easy to document which linux distro (and version) supports which system python method.
- It will also be easier use the white- and blacklist arguments ("omit" and "methods" of the keep.running())
- It will be easier to use the possible Method kwargs in the future (different kwargs for the system python version possible)
The Non-system-python versions of Methods having [SYS] version will obviously require additional package(s) installed in the current python environment. So, for example "org.freedesktop.PowerManagement" would require jeepney be installed on the current venv, while "org.freedesktop.PowerManagement [SYS]" would work if system python has jeepney installed.
Problem: Abrupt termination of parent process (SIGKILL, segfault, etc.)
I've done some testing on a solution which includes
- Parent process creating a client (a program entering wakepy mode)
- System python running in a subprocess, using wakepy Methods and communicating with a client
One problem I faced is that it's possible to end up in a situation where the subprocess ends up running there without a parent process forever, and potentially hogging 100% CPU. While it would be possible to use context managers or try .. except to fix most of the problems, those would not help in situations where the parent process is killed abruptly.
Should wakepy be ready for abrupt termination?
This is a good question. No context manager will help in case of segfault or similar. Examples:
- Python interpreter has a bug which causes segfault. Uncommon, but possible.
- Your code uses ctypes and you do something unexpected, like
ctypes.string_at(0) - Memory corruption in C extensions
- Stack overflow in C recursion
- OS level Out of Memory error
- Hard process termination by the OS, when python gets no time to react
Even though these are uncommon situations, these do happen. I don't think I would appreciate if a library I would be using would leave zombie processes in case of segfault. I would rather expect that everything is safe-guarded in such a way that users machines will not be left in an unclean, bad or unexpected state. In addition, one of wakepy's mottos or design principles is "Safe to crash". So yes, wakepy should be ready for abrupt termination and not leave zombie subprocesses behind.
Solution 1: prctl + parent death signal
One possible solution is to utilize prctl and basically set the kernel to send SIGKILL to the child process if the parent process dies with
prctl(PR_SET_PDEATHSIG, SIGKILL, 0, 0, 0)
This seems to be pretty much bullet proof solution to the problem. The following shows proof-of-concept python code to implement this:
PoC using prctl + PR_SET_PDEATHSIG
#main.py
import ctypes
import os
import subprocess
from pathlib import Path
if __name__ == "__main__":
script_dir = Path(__file__).parent
worker_script = script_dir / "worker.py"
parent_pid = str(os.getpid())
cmd = ["/usr/bin/python3.13", "-S", str(worker_script), parent_pid]
print(f"Starting subprocess with {cmd}")
proc = subprocess.Popen(
cmd,
text=True,
universal_newlines=True,
)
print(f"Subprocess started with PID: {proc.pid}")
print("Main process continuing with its own work...")
ctypes.string_at(0)
and
#worker.py
import os
import signal
import sys
import time
from ctypes import CDLL, get_errno
from ctypes.util import find_library
if __name__ == "__main__":
print("Worker script started.")
from ctypes import CDLL
real_parent_pid = int(sys.argv[1])
libc_path = find_library("c")
libc = CDLL(libc_path, use_errno=True)
PR_SET_PDEATHSIG = 1
res = libc.prctl(PR_SET_PDEATHSIG, signal.SIGKILL, 0, 0, 0)
if res != 0:
e = get_errno()
raise OSError(e, f"prctl failed: {e}")
parent_pid = os.getppid()
print(f"Worker script parent PID: {parent_pid}. Expected: {real_parent_pid}")
if parent_pid != real_parent_pid:
print("Parent process has died. Exiting worker.")
os.kill(os.getpid(), signal.SIGKILL)
print("Parent process still alive. Doing stuff.")
time.sleep(5)
print("Worker script exiting (done).")
Compatibility: This one is Linux only. On BSD, one should use procctl. There are also even smaller community OSes like the different flavors of SunOS/Solaris. Ideally wakepy would support everything, so if there's another (not too complicated) solution which is bullet proof and supports most of the systems, that should be used, instead. (Windows / macOS do not require a solution for using system python as wakepy is fully supported by using other methods). Other: This solution has also a possible problem with process ID recycling (a rare corner case, but still).
Solution 2: pipe-based parent-death detection
Idea: Use a pipe created by the parent process, with the read end passed to the child. The parent keeps the write end open. When the parent dies (normally or abnormally), the kernel closes the write end, causing the child’s read to return EOF (0 bytes). The child interprets this as “parent died.”
PoC using a lifeline pipe
#main.py
import ctypes
import os
import subprocess
import time
from pathlib import Path
if __name__ == "__main__":
rfd, wfd = os.pipe()
script_dir = Path(__file__).parent
worker_script = script_dir / "worker.py"
cmd = ["/usr/bin/python3.13", "-S", str(worker_script), str(rfd)]
proc = subprocess.Popen(cmd, text=True, universal_newlines=True, pass_fds=[rfd])
time.sleep(3)
ctypes.string_at(0) # cause a SIGSEGV
and
#worker.py
import os
import select
import sys
if __name__ == "__main__":
print("Worker script started.")
pipe_fd = int(sys.argv[1])
poll_interval = 1 # seconds
while True:
print("select.select()")
r, _, _ = select.select([pipe_fd], [], [], poll_interval)
if pipe_fd in r:
data = os.read(pipe_fd, 1024)
if not data:
print("Parent exited, exiting worker")
break
print("Exited worker!")
Compatibility: All POSIX systems; Linux, FreeBSD, OpenBSD, NetBSD, macOS, Solaris / Illumos, AIX, HP-UX, etc. (Not supported: Windows, WASI)
Next steps
Creating implementation of the ActivationServer and client using the pipe-based parent-death detection.