multi-core-python icon indicating copy to clipboard operation
multi-core-python copied to clipboard

Idea: Define an opt-in C-API subset usable with per-interpreter GIL

Open encukou opened this issue 3 years ago • 2 comments

The current C API is not compatible with per-interpreter GIL. For example, it contains global static types such as PyList_Type. We should definitely add functions to access these and deprecate the old way of doing things. However, that old way of doing things is widely used. Removing it from the C-API would be a major compatibility break – one that I think Python should not make.

Instead, I propose having two kinds of subinterpreters:

  • one that shares the GIL with the main interpreter and has no functional limitations,
  • and one with its own GIL, which can only load extension modules built without the problematic API, and can't start interpreters of the first kind.

This way, individual module authors can start opting in, starting with modules that would be most used (an useful) with separate GILs.

(But I'm in no rush to start working on this; lots of other issues – and converting the whole stdlib – can be tackled first.)

encukou avatar Feb 02 '21 21:02 encukou