spack
Add `+freethreading` variant to python (disables the GIL)
Python 3.13 adds a new configure argument --disable-gil, which builds Python without the Global Interpreter Lock. This option is currently experimental, and comes with performance tradeoffs for single-threaded tasks, but can cause big speedups for multithreaded workloads.
This adds a new variant +freethreading, which defaults to False but adds the --disable-gil argument when set to True.
The other way this could be done would be with a +gil variant with the default set to True (and install python~gil instead), but the official recommendation from the Python steering council to packagers is that the term for this type of Python should be "freethreading" when exposed to users, and that references to internal implementation details like the GIL should stay internal.
Based on this guidance from the Python Steering council I've renamed the variant to +freethreading after all.
I came here to do the same thing before noticing this PR (search didn't return anything immediately for some reason). I've also added +jit in the same PR (#47510). I'm happy to break these out into two separate PRs if that makes more sense.
But something I've noticed in the past two days building lots of variants of Python is that it seems Spack is not respecting the deeply-important differences in these builds when deciding if a previous build satisfies a dependency. That is, I created a Spack environment using +freethreading and then a second environment without it and Spack just used the previous build for the not-free-threaded variant. So I have two environments, py-313 and py-313t, both of which point to the free threaded build. I'm not sure what we can do or recommend to ensure Spack respects the difference.
@glentner did you request python~freethreading in the non-freethreading environment or did you let Spack choose for you? Spack will prioritize reuse of existing installs over variant defaults in the absence of a particular request about the variant value
@becker33 Ah, good advice. I bet that will fix the issue for me.
Reading more deeply into the feature I think HPC centers could build with both +jit and +freethread with a single install for Python and have PYTHON_JIT=0 and PYTHON_GIL=1 in their module by default. It seems these features are harmless even when included if you deactivate them at runtime.
Bouncing this closed/open to regenerate the merge commit and rerun the pipeline with a more recent develop branch
Hi @taliaferro, this package-related pull request can be migrated from spack/spack to spack/spack-packages using the migrate-pkg-prs utility.
We encourage authors to run the migration script themselves to preserve author attribution.
You can migrate all your open pull requests at once, following the steps in the documentation.
This comment was posted automatically.
