WIP: declare free-threaded support in pymodule macro
This is a WIP attempt to add a way to declare free-threaded support in the pymodule macro. I'm going on a trip soon so I'm opening this just in case someone else needs to finish it up before I get back.
Some wrinkles:
- Right now I'm only allowing one global setting per
pymodule. In principle you could add submodules that don't support free-threaded python to a parent module that does, but for now I think we can ignore that. Unfortunately it's tricky to thread the extra argument through down into where the actual FFI calls happen. - Not sure how to make modules added via
add_wrappedproperly inherit free-threaded support from the parent module for similar reasons. - How do we want to handle testing this?
- Should I mark all the existing modules in the pytests as supporting free-threading?
- If I don't do this, then under the free-threaded build the pytests will re-enable the GIL at runtime. Unfortunately it's an all-or-nothing thing.
- I can still test this PR by defining a module that says it needs the GIL, and then import it in a separate process so the runtime re-enabling doesn't "poison" the parent process.
- I added some code to the NumPy tests to make sure the GIL doesn't get re-enabled at runtime when new modules are added. I think the PyO3 pytests could probably just use that code.
- Alternatively, I could run the tests with
PYTHON_GIL=0, which means the GIL is never re-enabled at runtime no matter what metadata is attached to the module.- If I do it this way, there's no way to test that this PR actually does anything.
- Should I mark all the existing modules in the pytests as supporting free-threading?
Just a note that this should probably only be merged once all the "RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module foo" warnings in the python tests are fixed in the free-threaded build.
Just a note that this should probably only be merged once all the "RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module foo" warnings in the python tests are fixed in the free-threaded build.
This is done now and I think all the tests are passing so I'm going to move this out of draft phase and update the PR description.
This probably needs some discussion in the new guide section on free-threading support.
woohoo! looks like tests are passing except for the codecov issue affecting all PRs right now.
It occurred to me while working on something else that this PR should remove the UNSAFE_PYO3_BUILD_FREE_THREADED uses in the CI scripts a well.
EDIT: done