Python 3.13 free-threading compatibility

[lysnikolaou]

Hey all! 👋

This aims to be a tracking issue for all work necessary to support the free-threaded build of Python 3.13. A very high-level list of steps is (more details to follow as I investigate more issues):

• [x] Set up CI for free-threading
  • [x] https://github.com/python-pillow/Pillow/pull/8200
• [x] Move away from APIs that use borrowed references under the free-threaded build
  • [x] https://github.com/python-pillow/Pillow/pull/8216
• [x] Audit C extension module for thread-safety issues
  • [x] #8238
• [x] Mark C extension modules as thread-safe with the Py_mod_gil slot (or PyUnstable_Module_SetGIL)
  • [x] https://github.com/python-pillow/Pillow/pull/8216
• [x] Upload wheels for the free-threaded build. Other projects have started uploading nightlies for the free-threaded on the scientific-python-nightly-wheels index so that we can enable faster iteration, we can discuss whether we wanna do this for Pillow as well.
  • [X] https://github.com/python-pillow/Pillow/pull/8219
  • [x] https://github.com/scientific-python/upload-nightly-action/issues/84
  • [x] #8236
  • [x] https://github.com/python-pillow/Pillow/pull/8243
  • [x] https://github.com/python-pillow/Pillow/pull/8244

[hugovk]

Thanks for opening this issue!

As some background, the acceptance of PEP 703 lays out the three phases for disabling the GIL. Python 3.13 is the first in the experimental phase I, where a free-threaded build is available as an option in the Windows and macOS installers, and can be built for Linux (and the other platforms) using an extra flag.

So now is a good time to start testing on free-threaded builds, and getting the code ready. It will be good to produce wheels at some point, to help others test and integrate. These will be experimental and not for production.

I've made a start on the first two steps, I'll open a PR for the CI first.

Some handy references:

• https://dev.to/hugovk/help-us-test-free-threaded-python-without-the-gil-1hgf
• https://docs.python.org/3.13/howto/free-threading-extensions.html
• https://github.com/Quansight-Labs/free-threaded-compatibility

[hugovk]

Please see PR https://github.com/python-pillow/Pillow/pull/8200 to set up CI for free-threading.

[hugovk]

I'm working on this today with @lysnikolaou at the EuroPython sprints.

It would help the ecosystem if we also upload nightly wheels so others can test the current code without needing to wait for quarterly releases.

@lysnikolaou has requested access at https://github.com/scientific-python/upload-nightly-action/issues/84 for us to upload to the Scientific Python Nightly Wheels index, where other projects like SciPy are uploading.

See also https://labs.quansight.org/blog/free-threaded-python-rollout and https://py-free-threading.github.io/.

[radarhere]

If anyone would like wheels generated after #8216, here you go - Linux.zip, macOS.zip, Windows.zip

[lysnikolaou]

Nightly wheels for Linux, macOS and Windows are now available on https://anaconda.org/scientific-python-nightly-wheels/pillow/.

[rgommers]

https://anaconda.org/scientific-python-nightly-wheels/pillow rather than pandas. And, yay thank you both! 🎉

[radarhere]

Every task in this issue has been completed - is this just open now as a tracking issue until Python 3.13 is finalized?

[lysnikolaou]

is this just open now as a tracking issue until Python 3.13 is finalized?

Yes, I'd leave it open until at least the final release of 3.13 and see how it goes from there.

[radarhere]

Python 3.13 has now been released - https://www.python.org/downloads/release/python-3130/

[radarhere]

Pillow 11.0.0 has now been released, with 3.13t wheels.