nltk
Test free-threaded Python with pytest-run-parallel
To be confident about free-threaded compatibility, it is recommended to test with pytest-run-parallel.
- https://py-free-threading.github.io/testing
Something like:
% uvx --with=pytest-run-parallel pytest --iterations=8 --parallel-threads=auto
@purificant @MrMarvel @ekaf
Update 2025_11_09:
- [x]
import sklearncurrently re-enables the GIL but a fix is in progress...- https://github.com/scikit-learn/scikit-learn/pull/32528#issuecomment-3506086127
- [ ] 37 test functions are currently marked as thread-unsafe.
@pytest.mark.iterations(1) @pytest.mark.parallel_threads(1)
Perhaps we are not free threading compatible...
% uvx --with=scikit-learn python3.14t -c 'import sklearn'
<frozen importlib._bootstrap>:491: RuntimeWarning: The global interpreter lock (GIL) has been enabled
to load module 'sklearn.__check_build._check_build', which has not declared that it can run safely
without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with
PYTHON_GIL=0 or -Xgil=0.
Maybe at this point, no classifier for free threading is better, or
Programming Language :: Python :: Free Threading :: 1 - Unstable
as per https://py-free-threading.github.io/porting/#pytest-is-not-thread-safe
Some of the 37 thread-unsafe tests may be related to LazyCorpusLoader’s load/unload race patterns and may be stabilized by PR #3460; the remainder likely involve other shared mutable state or resource lifecycle concerns elsewhere in the codebase.
Agreed. The question is which PR do we merge first, and then how do we step through fixing the remaining issues?
I have blocked both:
@pytest.mark.iterations(1) @pytest.mark.parallel_threads(1)
But perhaps only one is needed for each case.
What is the recommended way to do localhost (M1 MacBook) testing so I do not need to thrash the GitHub Actions servers?
What is the recommended way to do localhost (M1 MacBook) testing so I do not need to thrash the GitHub Actions servers?
There's tox in the codebase, it's a little undermaintained since we started relying on CI more and evolving it in preference of other testing setups, some improvements might be needed to streamline it.
Why did you close this PR @cclauss? As you wrote, it does not overlap with #3460, so we will still need to find and fix the thread-unsafe tests.