benfred
Detect GIL on python 3.12+
Python 3.12 moved from having the GIL held in _PyRuntime.tstate_current to being in thread local storage https://github.com/benfred/py-spy/issues/633#issuecomment-1815273709 . This means that we currently can't detect which thread is holding the GIL
also worth mentioning: deadsnakes is already building python3.13-nogil, meaning you'll have a /usr/bin/python3.13-nogil executable which will run with GIL disabled
on the nogil build you can turn it on and off for instance via env var ref https://github.com/python/cpython/blob/v3.13.0/Lib/test/test_cmd_line.py#L882
add-apt-repository ppa:deadsnakes/ppa
apt-get update
apt-get install python3.13-nogil
To check my understanding - this means that results can be quite skewed in multithreaded programs right where more than one thread does significant work? If two threads are working in function f and g respectively both f and g will accumulate work/samples each time py-spy samples?
To check my understanding - this means that results can be quite skewed in multithreaded programs right where more than one thread does significant work? If two threads are working in function f and g respectively both f and g will accumulate work/samples each time py-spy samples?
No - the idle detection code in py-spy should prevent skewing the results here . Not having GIL support in python 3.12 means that we can't use the --gil flag in py-spy to only profile threads with the GIL - not that the results will be less accurate in multi-threaded programs when not using the --gil flag
Similarly, uv supports installing freethreaded Python distributions that could be used for testing here.
No - the idle detection code in py-spy should prevent skewing the results here . Not having GIL support in python 3.12 means that we can't use the --gil flag in py-spy to only profile threads with the GIL - not that the results will be less accurate in multi-threaded programs when not using the --gil flag
Ah, that's great! So the only downside is more sampling overhead in some cases :ok_hand:
Fwiw - fixing this turned out to be easier than I initially thought (We can read the gil_runtimestate from the PyInterpreterState directly), and this feature will be much easier to maintain for future python releases