tiktoken icon indicating copy to clipboard operation
tiktoken copied to clipboard

Add support for free-threaded CPython 3.13t

Open rostan-t opened this issue 6 months ago • 0 comments
trafficstars

This PR proposes adding support for free-threaded Python.

This requires bumping PyO3 to a newer version as support was added in version 0.23.0. If I'm not missing anything, the library is already thread-safe so no other change is required apart from fixing warnings introduced by PyO3 0.23.

Note: Unfortunately, regex doesn't declare free-threading support and there doesn't seem to be a plan to do so^1. Disabling the GIL therefore still requires passing setting PYTHON_GIL=0 or -Xgil=0.

rostan-t avatar May 13 '25 17:05 rostan-t