[ENH] Add a way to set Py_mod_gil slot for modules defined in cython
The free-threaded python build requires that modules explicitly declare whether or not they support running without the GIL using either PyUnstable_Module_SetGIL for single-phase init or the Py_mod_gil slot for multi-phase init.
If the slot isn't set, python warns at runtime about this and re-enables the GIL, unless you force the interpreter to disable the GIL:
In [1]: from numpy import *
<frozen importlib._bootstrap>:488: RuntimeWarning: The global interpreter lock (GIL) has been enabled to
load module 'numpy.random.bit_generator', which has not declared that it can run safely without the GIL.
To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.
I tried looking at doing that from inside the definition of a cython module in the cython language, and I came to the conclusion that cython needs to handle this in the compiler, since the changes we need to do to the module need to be in the module definition in C or C++.
I don't think cython can just automatically say that modules rely on the GIL or not, since code may implicitly be locking around the GIL. It makes sense to me to let users explicitly opt-in with a global or module-local compiler directive.
That makes sense to me.
I can get started on implementing this.
I assume this would need both a source-level directive and a front-end flag for cython/cythonize?
For NumPy and SciPy I'd expect we would want to use it as cython --free-threaded-safe (or whatever the flag ends up being called).
Souce-level directives are automatically command line arguments to the cython command via -X so I don't think anything special needs doing there. (I can't remember about cythonize off the top of my head).
I'm not quite sure why you'd want to use the command-line argument as most of the time though? It seems more like something the source code should know rather than the the build system. (It would be a no-op for non-free-threaded Python builds so wouldn't break anything to set it universally once you'd decided that an individual file was good).
Ah yes, good point about -X. Nice and easy then:)
CLI arguments: because (a) easier to audit since we have way more .pyx files than build config files, and (b) a cleaner end state (a single flag somewhere, right next to -3) rather than an annotation in every single source file. The source file annotations are also easy to forget when a contributor adds a new .pyx file.
Minor logistical request: could this issue and the linked PR please be labelled with nogil CPython?