uvloop icon indicating copy to clipboard operation
uvloop copied to clipboard

Add support for free-threaded (nogil) 3.13 Python

Open gi0baro opened this issue 1 year ago • 2 comments

SSIA

This also definitely depends on future Cython 3.1.0 release

gi0baro avatar Nov 07 '24 17:11 gi0baro

Cython 3.1.0 is out so someone should try to build it and run the tests.

See https://py-free-threading.github.io for more info. I'm also happy to help out.

ngoldbaum avatar May 19 '25 18:05 ngoldbaum

Builds fail:

  uvloop/loop.c:4537:80: error: unknown type name ‘__pyx_vectorcallfunc’; did you mean ‘vectorcallfunc’?
   4537 | static CYTHON_INLINE PyObject *__Pyx_PyVectorcall_FastCallDict(PyObject *func, __pyx_vectorcallfunc vc, PyObject *const *args, size_t nargs, PyObject *kw);
        |                                                                                ^~~~~~~~~~~~~~~~~~~~
        |                                                                                vectorcallfunc

Full error is here as well: https://ft-checker.com/?search=uvloop

It would be amazing to have free threaded python supported by uvloop, its one of the few remaining packages that is yet to support free threading :)

WebShark025 avatar Jun 11 '25 01:06 WebShark025

Not sure why ft-checker was seeing an error back in June, but it looks like with the latest version of Cython, uvloop builds successfully:

goldbaum at Nathans-MBP in ~/Documents/uvloop on master
± python -m pip install -e .
Obtaining file:///Users/goldbaum/Documents/uvloop
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: uvloop
  Building editable for uvloop (pyproject.toml) ... done
  Created wheel for uvloop: filename=uvloop-0.21.0-0.editable-cp313-cp313t-macosx_15_0_arm64.whl size=9603 sha256=5ce81ea00fdfa677b1aea648fa00265cdea23a0bb7aaaea89679a3fbc4964a4f
  Stored in directory: /private/var/folders/nk/yds4mlh97kg9qdq745g715rw0000gn/T/pip-ephem-wheel-cache-3cc8t1nw/wheels/82/ff/6a/4d53e60cdf47a728d2d4c0e034ecb7018204cbcedf4763990e
Successfully built uvloop
Installing collected packages: uvloop
Successfully installed uvloop-0.21.0

goldbaum at Nathans-MBP in ~/Documents/uvloop on master
± python -VV
Python 3.13.5 experimental free-threading build (main, Jul 13 2025, 10:48:09) [Clang 17.0.0 (clang-1700.0.13.5)]

uvloop has a test dependency on PyOpenSSL, which in turn depends on cryptography, which in turn depends on CFFI, which doesn't yet ship a version supporting the free-threaded build. With a hacked-together environment based on my CFFI PR adding free-threaded support, all but one of the tests pass, and the failure looks unrelated to free-threading:

========================================= short test summary info ==========================================
FAILED tests/test_aiohttp.py::Test_UV_AioHTTP::test_aiohttp_graceful_shutdown - TypeError: TCPSite.__init__() got an unexpected keyword argument 'shutdown_timeout'
===================== 1 failed, 473 passed, 34 skipped, 1 warning in 77.59s (0:01:17) ======================

If I force each test to run many times in a thread pool using pytest-run-parallel, I see no additional failures. That's a great sign that the wrappers and libuv itself don't implicitly rely on global state, at least to the extent the test coverage is good. pytest-run-parallel is only good for checking for global state in the implementation, it can't detect thread safety issues due to mutable objects, I don't know offhand if uvloop needs to worry about that. I do see there are explicitly multithreaded tests, which is also good.

We can probably come back to this once cryptography and PyOpenSSL support the free-threaded build.

ngoldbaum avatar Jul 17 '25 15:07 ngoldbaum

I still get the same build error as highlighted in ft-checker with:

Python 3.13.5 experimental free-threading build | packaged by conda-forge | (main, Jun 16 2025, 08:23:46) [Clang 18.1.8 ]

dandiep avatar Aug 13 '25 21:08 dandiep

You'll need to build uvloop from a checkout of the repository. The uvloop 0.21.0 source release on PyPI bundles cython code that was generated with a release of Cython that does not support the free-threaded build.

ngoldbaum avatar Aug 13 '25 21:08 ngoldbaum

Can confirm, that works. Thanks!

dandiep avatar Aug 13 '25 21:08 dandiep

Fixed in #693

fantix avatar Oct 15 '25 01:10 fantix