Support for building `python-tcod` with Python 3.13 `--disable-gil` (no-GIL) build

[coldwarrl]

Hi there 👋

First off, thank you for maintaining python-tcod — it's a really valuable library for roguelike development!

I've been experimenting with Python 3.13's new --disable-gil feature (based on the free-threaded no-GIL branch), and I wanted to see how well python-tcod builds and works in that environment.

🛠 What I tried:

• Cloned the repository and initialized submodules.

• Patched build_libtcod.py to conditionally skip Py_LIMITED_API when Py_GIL_DISABLED is detected via sysconfig.get_config_var("Py_GIL_DISABLED").

• Removed py-limited-api entries from pyproject.toml.

• Attempted to install in editable mode with:

  pip install -e .
  

---

⚠️ What fails:

The build still fails due to low-level C header issues during the CFFI extension build:

error: "The limited API is not currently supported in the free-threaded build"
error: unknown type name 'PyMutex'
error: call to undeclared function '_Py_atomic_*'

---

🧠 My understanding:

It seems that, despite disabling Py_LIMITED_API, some core Python headers or assumptions in libtcod or tcod._libtcod still expect GIL-based behavior and are not currently compatible with the free-threaded interpreter.

---

🤛 Feature Request:

Would it be possible to support building python-tcod on GIL-disabled Python versions in the future?

Even limited or experimental support (e.g. via conditional compilation) would be amazing for those of us exploring Python's no-GIL future.

But I cannot judge the effort.

[HexDecimal]

I'm interested in free-threaded no-GIL builds.

From what little research I've done, I suspect the issue is in CFFI itself and that issue might've already been fixed but not in an official release. So I suspect that you don't need to remove references to Py_LIMITED_API from tcod's build files, instead you need to fetch and build cffi directly from its latest source and then tcod will build normally.

Otherwise I'll need to install a GIL-less version of Python on the GitHub workflows in order to verify any fixes.

[coldwarrl]

I tried

pip install git+https://github.com/python-cffi/cffi.git

but did not help if you have meant that ...

[HexDecimal]

I tried

pip install git+https://github.com/python-cffi/cffi.git

but did not help if you have meant that ...

I did mean that. I'm unsure what to try next.

[coldwarrl]

clang -UNDEBUG -UPy_LIMITED_API ... /Users/xxx/.pyenv/versions/3.13-dev/include/python3.13t/Python.h:51:4: error: "The limited API is not currently supported in the free-threaded build"

somehow it remains, although I changed the build script.

This limitied api is only needed due to older python versions, right ? Probably you do not want to discard those...

[HexDecimal]

I assume you've also removed the py_limited_api=True line from build_libtcod.py?

[coldwarrl]

yes

py_limited_api=not sysconfig.get_config_var("Py_GIL_DISABLED"),

i will look again tomorrow...maybe AI has also some further hints

[coldwarrl]

this is what the AI analzed:

⚠️ Python-tcod Build Issue with Python 3.13 No-GIL

Problem Summary

Even though the project no longer sets Py_LIMITED_API in the ffi.set_source call, building python-tcod against Python 3.13 (No-GIL variant) still triggers errors related to the limited API. Specifically, compilation fails with:

#error "The limited API is not currently supported in the free-threaded build"

Why This Happens

1. CFFI Always Includes <Python.h>

   • CFFI-generated .c files (like tcod._libtcod.c) always include the Python C API via #include <Python.h>, regardless of your build flags.

2. Python 3.13 No-GIL Is Strict About the Limited API

   • Python 3.13’s headers explicitly disallow use of the limited API in no-GIL builds.
   • Even without Py_LIMITED_API being defined, some internal logic or missing defines can still trigger the "limited API" checks.

3. The Issue Is Triggered Inside Python’s Headers

   • This is not due to an explicit mistake in your own code, but due to changes in Python’s internal headers and how CFFI-generated code interacts with them.

Workarounds and Suggestions

• Use the standard (GIL-enabled) CPython 3.13 to build python-tcod, if you're not specifically targeting No-GIL compatibility.
• Add a check in build_libtcod.py to detect unsupported Python builds and fail early with a clear message.
• Investigate CFFI-level solutions: If possible, suppress or replace #include <Python.h> in the generated C code (this is non-trivial).
• Upstream Watch: Monitor the status of CFFI and CPython 3.13 development to see if proper No-GIL compatibility will be added.

Conclusion

This issue isn’t caused by incorrect flags in your project, but by deeper incompatibilities between how CFFI-generated files interact with the Python C API headers in No-GIL builds. A future-proof fix will likely require changes in either Python itself or CFFI.

---

Sounds reasonable to me, but i cannot judge the cffi/c stuff...

A workaround for my project is now to use Rust for multi-core algorithms and call it from python...should be doable, I guess

[HexDecimal]

This is not due to an explicit mistake in your own code, but due to changes in Python’s internal headers and how CFFI-generated code interacts with them.

There is a blatant lack of specificity here from the AI. The AI seems to state stuff that is relevant but is unable to properly explain how everything interacts with each other.

CFFI needs to include #include <Python.h> in its generated code _libtcod.c. The issue is that CFFI might set Py_LIMITED_API before it includes that header. Looking at _libtcod.c shows how this actually happens as well as some minor documentation on how CFFI uses Py_LIMITED_API.

What I notice is that CFFI internally uses _CFFI_NO_LIMITED_API to disable adding Py_LIMITED_API. Perhaps you can try adding a define for _CFFI_NO_LIMITED_API which should prevent it from defining Py_LIMITED_API in the generated code.

[coldwarrl]

I have tried that also but no success....so I guess there is no easy solution for now...

[rumly111]

you realize that tcod is used to create roguelikes? why would you care about gil/no-gil with turn-based games?

[HexDecimal]

you realize that tcod is used to create roguelikes?

Experimental implementation features are exactly the kind of thing I'd expect roguelike developers to care about. It'd be fun to mess around with disable-GIL even if it ends up not having much of an impact on performance. I like to be on the cutting edge when its possible.

Note that tcod already releases the GIL for its biggest algorithms, so one can already run multiple threads in some cases.

why would you care about gil/no-gil with turn-based games?

Glossing over that libtcod has been used for real-time games before. There are cases where so much computation can happen on a single turn that there could be a real benefit from running multiple Python threads, especially relating to AI.

That said there's not much I can do about this until things are resolved upstream in the Python cffi package.