Status of Conda ecosystem support

[rgommers]

• [x] Python itself and pip are now available in the ad_testing channel (builds on top of defaults): https://py-free-threading.github.io/installing_cpython/#conda
• [x] conda-forge support (work not started yet)
• [x] Issue with installing noarch packages in conda (& other installers): https://github.com/conda/conda/issues/14053

Currently (as of July 23rd, 2024) the recommended way to set up a conda dev environment for free-threaded CPython is to install python and pip from the ad-testing channel, and then install everything else from source or from nightly wheels.

[jjhelmus]

Conda-forge has a free-threading release of Python 3.13.0rc1: conda-forge/python-feedstock#679.

[lesteve]

FWIW the conda-forge command to install a free-threaded CPython from conda-forge was added in #74 and is available in https://py-free-threading.github.io/installing_cpython/#conda-forge

[rgommers]

Conda-forge has had packages for a month or so now for both Python itself and for some key packages like numpy and cython (thanks to work by @isuruf). The noarch issue was also resolved, and a migrator is waiting to get started with larger-scale rebuilding once more projects have done a free-threading compatible release: https://conda-forge.org/status/migration/?name=python313t. So we're all good here for now.

I did a bit of automation to set up a Pixi dev environment for SciPy, that should translate pretty well to other packages with compiled code. If anyone needs it (extracted and cleaned up a bit from https://github.com/rgommers/pixi-dev-scipystack/blob/main/scipy/pixi.toml):

[feature.free-threading]
channels = ["conda-forge/label/cython_dev", "conda-forge"]

[feature.free-threading.dependencies]
python-freethreading = ">=3.13.0,<3.14"
compilers = ">=1.7.0,<2"
pkg-config = ">=0.29.2,<0.30"
ninja = ">=1.12.1,<2"
ccache = ">=4.10.1,<5"
meson = ">=1.6.0,<2"
meson-python = ">=0.16.0"
cython = ">=3.1.0a0,<4"
pythran = ">=0.17.0"
python-build = "*"
pip = "*"
blas-devel = "*"
numpy = ">=2.1.3"
pybind11 = ">=2.13.1"
pytest = "*"
hypothesis = "*"
pytest-run-parallel = "*"

[feature.free-threading.tasks]
build-nogil = { cmd = "your-build-cmd", env = { CC = "ccache $CC", CXX = "ccache $CXX" } }
test-nogil = { cmd = "your-test-cmd" }

[lesteve]

a migrator is waiting to get started with larger-scale rebuilding once more projects have done a free-threading

Just curious, do you have some kind of rough timeline for the conda-forge migration to start? Are there some key packages that you think should have a release with preliminary [^1] free-threaded support before the migration is launched on conda-forge?

I have read https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/6673 and I can see there is a fine line to thread between making free-threaded available to users so they have a chance to test and report issues vs adding one more thing to do for busy conda-forge maintainers ...

[^1]: numpy 2.1 wording in their release notes which I used in scikit-learn changelog as well 😉)

[rgommers]

@lesteve it's up to the conda-forge core team, but I'd wait at least for the widely used code generators (Cython 3.1, Cffi) and some key packages with say >5,000 direct dependencies according to https://deps.dev/ (e.g., SciPy, scikit-learn, Matplotlib, Cryptography, Pydantic, ...). And the counter at https://hugovk.github.io/free-threaded-wheels/ creeping quite a bit closer to 360 (now 328/360).

Probably end of Q1'25 is the earliest that can happen. In the meantime, packages can be built by hand if you know they're compatible, which can be helpful if you're just needing that one dev dependency to make your life easier:)

[isuruf]

For proper mamba support, we need https://github.com/mamba-org/mamba/pull/3579 (EDIT: merged now)

[h-vetinari]

This would IMO also be good to fix before a larger-scale roll-out: https://github.com/conda/conda-build/issues/5563