docker-library
Is there a 3.13 nogil container?
I want a 3.13 nogil container, also with all optimizations, etc. I assume that the current 3.13 containers use the GIL.
We don't have a "nogil" image currently. For 3.13 we're using the default compilation settings, which AFAIK still results in the GIL. Unfortunately, compiling Python is pretty heavy, so expanding our current support matrix to include new "nogil" variants is also going to be a tough sell. :see_no_evil:
Are you saying you don't compile Python?
I'm not sure how you got there from what I said -- perhaps I can clarify: we do compile Python (currently ~42 times per architecture, in fact, and we support ~10 architectures where compilation is necessary), and it's very heavy to do so, and adding more variants would require us to do so even more times, so is not something we're willing to consider unless/until it is an officially recommended configuration (ideally the default configuration, but we'd be willing to consider an official statement along the lines of "everyone should try this out").
nogil is the future of Python. It doesn't make sense to not support it. If this request seems strange, that may be so only because I am the first person to ask for it. Rest assured, there will be many more who will progressively ask for it. The pressure will build. This request is easy to reject now, but only now.
I agree that nogil is the future of Python, and there's nothing strange about the request. If I were completely opposed to the idea in general, I probably would've closed the issue already. It's more a matter of resourcing/prioritization, not desire/understanding. If the demand were actively higher, it would also be a lot easier to justify the added resources/maintenance spend -- the best way for folks to signal that is to react with :+1: on the top post in this thread.
To put that another way, my answer is more accurately "not yet", not "never".
If you'd like to help with the work, figuring out what changes are necessary to our current Dockerfiles would be a really useful first step.
Instead of prodiving a whole second set of "nogil" Docker images the free-threaded executable could also be part of the normal images just like it is done with the official Windows and macOS installers:
CPython now has experimental support for running in a free-threaded mode, with the global interpreter lock (GIL) disabled. This is an experimental feature and therefore is not enabled by default. The free-threaded mode requires a different executable, usually called
python3.13torpython3.13t.exe. Pre-built binaries marked as free-threaded can be installed as part of the official Windows and macOS installers, or CPython can be built from source with the --disable-gil option.
https://docs.python.org/3.13/whatsnew/3.13.html#free-threaded-cpython
So we would have /usr/local/bin/python3.13t in every image and call this to use the free-threaded mode.
Unfortunately that still means that all of Python has to be compiled twice.
@sekrause But that would break a lot of tooling that expects Python to be at the prior file path.
But that would break a lot of tooling that expects Python to be at the prior file path.
The normal paths would still be there, but point to the normal GIL version. So if you don't adjust your tooling everything will be as before.
If anyone seeking for some basic image to play with, I've made one: https://github.com/abebus/free-threaded-python-docker-image
Python images without GIL and JIT enabled should be part of docker images. I have been using docker as a dev container and to experiment new versions of python. Thanks to all the work been done so far.
I'm a bit surprised that you're going out of your way to provide images for alpha versions of Python (if I'm reading https://hub.docker.com/_/python correctly, there are images for Python 3.14.0a2), but not for free-threaded builds which are an official feature of released Pythons. While these builds are still marked experimental, many Python packages have started using them to ensure they are free-threading compatible.
For example in Apache Arrow we have our own image for CI until the official Python Docker images support free-threaded builds: https://github.com/apache/arrow/blob/main/ci/docker/python-free-threaded-wheel-manylinux-test-imports.dockerfile
Many Python open source projects are probably using similar home-grown recipes due to lack of official Docker image availability.
I think four builds are needed:
- GIL and noJIT (current build)
- noGIL and noJIT (this issue)
- GIL and JIT
- noGIL and JIT
Is this correct?
Those are unrelated, perhaps open a separate issue for JIT builds?
(but I'd argue that JIT builds are unimportant for now, as they don't provide tangible benefits)
Those are unrelated, perhaps open a separate issue for JIT builds?
It matters because the naming convention has to account for all possible variations. Also, it might help in understanding that bundling multiple versions in the same image is not a great idea.
(but I'd argue that JIT builds are unimportant for now, as they don't provide tangible benefits)
JIT may not matter too much now, but it's the sauce that could get Python closer to Julia-like speeds, so it may matter in the long term.
https://github.com/docker-library/python/issues/947#issuecomment-2284513473 is still an accurate representation of the current maintainer position
I would be willing to consider something like https://github.com/docker-library/tomcat/pull/299 to have a place where folks can see/test Dockerfiles for this, but we cannot reasonably commit to producing official built images for it at this time.
While it's not exactly a fun time to roll your own image, most of the work is done by the team here. I built my own image pretty quickly to do some freetheading experiments and documented how I did it here: https://github.com/yaml/pyyaml/issues/856#issuecomment-2807211962
You could also enable experimental JIT fairly quickly with yet another option (--experimental-jit-interpreter) as documented upstream
Rather than building Python yourself, a better option would be to use one of the builds from https://github.com/astral-sh/python-build-standalone/
Rather than building Python yourself, a better option would be to use one of the builds from https://github.com/astral-sh/python-build-standalone/
I suppose this might be usable from uv setting up a project, and that the Dockerfile will have to use uv in this way, specifically by defining 3.13t or such (with the t suffix) as the Python requirement. I haven't tried it.
There are some open issues/bugs though wrt uv's support for t Python.
Various other ways to instal free-threaded Python are listed at https://py-free-threading.github.io/installing-cpython/, also via docker, such as perhaps via the quay.io/pypa/manylinux_2_28_x86_64 image.
Given the significant improvements to the free-threaded mode in Python 3.14, I would suggest prioritizing #1052 over providing a container for free-threaded 3.13.
@tianon You mentioned that you would consider this further if there was an official announcement encouraging the community to use the free-threaded build. Recently, PEP 779 was approved, moving the free-threaded build to Phase II (not experimental, but not default).
The Steering Council announcement about the PEP acceptance specifically says:
With these recommendations and the acceptance of this PEP, we as the Python developer community should broadly advertise that free-threading is a supported Python build option now and into the future, and that it will not be removed without following a proper deprecation schedule. As approved by the 3.14 Release Manager Hugo van Kemenade, we recommend officially removing the “experimental” tag from the CPython free-threading build with the 3.14 beta 3 release.
Do I understand correctly that you're currently compiling Python over 400 times for the various architectures, and that supporting the free-threaded build would double this?
Do I understand correctly that you're currently compiling Python over 400 times for the various architectures, and that supporting the free-threaded build would double this?
Yes, that is correct in the abstract; in the specific, the various holes in the matrix mean it's closer to ~200 (232 at present, will increase when, for example, Trixie is released and #1042 means we add a bunch of notoriously slow/heavy-to-build riscv64 images, for example), but that's still a ton and it honestly kind of hurts to see it written out explicitly.
We could shrink the impact by doing "clever" things like only supporting "nogil" on a limited set of architectures, a limited set of base distributions, etc, but ultimately any choice we make there is definitely going to cause some amount of friction (both for us in maintaining and explaining the line, and for users who get to navigate the line).
We could also consider just including both in the default images, but I doubt that's going to be a popular choice either (and doesn't decrease the compilation burden, just obfuscates it). Ultimately it would be really interesting if this could be an entirely runtime determined value (or some way of having two binaries compiled concurrently and using linking cleverly so that they share most of the final binary size in shared libraries), but that's probably also a tall order.
(All of this is also not getting into the topic of what to call the tags/variants, because that's a whole separate rabbit hole as evidenced by that PEP using terms like "GIL-less", "free-threaded", and "with-GIL" and never using "nogil" to describe this. 🙈)
Yes, that is correct in the abstract; in the specific, the various holes in the matrix mean it's closer to ~200 (232 at present
Is this the count for each version of Python, e.g. 3.13, or the sum across all supported versions?
currently ~42 times per architecture, in fact, and we support ~10 architectures where compilation is necessary
(I said this previously, which is where the 400 ballpark number came from)
3.13 is currently 40 and 3.14 is 38.
Ultimately it would be really interesting if this could be an entirely runtime determined value
Technically, it is possible to re-enable the GIL on the free-threaded build by setting the environment variable PYTHON_GIL=1. I don't know if it's intended for something like this. If you want, I could try getting in touch with the core developers working on the free-threaded build and see if they have any input.
The caveat there is that historically, the nogil Python was thought to be a little slow for the single-threaded case than the gil python. I don't know how far they've come in covering this gap.