giampaolo
Add wheels for the free-threaded CPython build & test on CI
Summary
- OS: All OS's
- Type: core, wheels
Description
Currently, psutil only offers wheels for the stable ABI. Ideally, we should also provide wheels for the free-threaded CPython build and also test it in CI.
There's also some global state in various places that needs to be locked in case it's intended to be used from multiple threads.
Note that this was already proposed in gh-2491, with a lot of relevant discussion. The key comment is https://github.com/giampaolo/psutil/pull/2491#issuecomment-2591265995 (wait for abi4). That decision seems a little unfortunate (but it's early days); psutil is quite heavily used, so for CPython 3.14 in a few months it seems like cp314t wheels will help a lot of users. abi4 won't arrive until Python 3.15. Our team will be happy to contribute the extra maintenance effort that may be required to build and support cp314t wheels.
There's also some global state in various places that needs to be locked ...
What global state is there?
What global state is there?
I only spent a little bit of time on this. In that time I could find the following places:
-
PSUTIL_DEBUG: Can probably stay as is, people shouldn't really be changing that across threads, I think. -
PSUTIL_CONN_NONE: This is a constant and probably doesn't need changing. -
psutil_xfiles&psutil_nxfiles: These appear to be a list of open files populated by a syscall. Probably need locking. -
kihead&kpcbhead: Again, these look like linked lists of open files & sockets. Probably need locking. -
PSUTIL_MACH_TIMEBASE_INFO: Only initialized once at module initialization, so not problematic. -
PSUTIL_WINVER,PSUTIL_SYSTEM_INFO&PSUTIL_CRITICAL_SECTION: Only initialized once at module initialization, so not problematic. -
globalFileName: Does not need changing, because a critical section is already used to access it. -
TimeoutExpired&TimeoutAbandoned: Only initialized once at module initialization, so not problematic. -
g_TcpTableSize&g_UdpTableSize: These look like counters of open sockets. Potentially problematic. -
load_avg_1m,load_avg_5m&load_avg_15m: Not sure what these are, but they seem potentially problematic as well.
Thanks, that's helpful. For things like psutil_xfiles, I think we should make the state local to the function call (psutil_net_connections) instead of adding locking.
For things like that (psutil_xfiles) feel free to open individual PRs (actually it's better).
All of the PRs fixing places where global state is used have been merged. I've opened #2609 to build cp313t wheels. There's one test failure that seems to be unrelated to the changes we did (it fails while building cp313 (gil-enabled) wheels).
psutil.users() on UNIX systems also has some issues: https://github.com/giampaolo/psutil/issues/2614.
#2614 has been resolved, so I think #2609 is next to build free-threaded wheels.
I reverted https://github.com/giampaolo/psutil/pull/2590 since it was causing a segfault.
py314t wheels added in https://github.com/giampaolo/psutil/pull/2659. I'll publish a release soon.
Blog post. @lysnikolaou I took the liberty to mention you: https://gmpy.dev/blog/2025/wheels-for-free-threaded-python-now-available-in-psutil