cpython icon indicating copy to clipboard operation
cpython copied to clipboard

C API: Add PyType_GetModuleByDef() to the limited C API version 3.13

Open vstinner opened this issue 1 year ago • 6 comments

I propose adding PyType_GetModuleByDef() to the limited C API version 3.13. The function was added to Python 3.9, the function is now well tested. We could add it way earlier to the limited C API.

cc @encukou


The PyType_GetModuleByDef() function is needed to access the module state in C function which don't get a module as first parameter, but only an instance of a heap type. Heap types should be created with PyType_FromModuleAndSpec(module, spec, bases) to store the module in the type. PyType_GetModuleByDef(type) gives the module and then PyModule_GetState() gives the moulde state.

Examples of functions which don't get a module argument:

  • Class methods
  • Most slots, examples: tp_new, tp_richcompare, tp_iternext, etc.
  • PyGetSetDef getter and setter functions

Without PyType_GetModuleByDef(), many static types cannot be converted to heap types.

The following stdlib extensions use PyType_GetModuleByDef()

Modules/_asynciomodule.c
Modules/_bz2module.c
Modules/_collectionsmodule.c
Modules/_csv.c
Modules/_decimal/_decimal.c
Modules/_elementtree.c
Modules/_functoolsmodule.c
Modules/_io/_iomodule.h
Modules/_pickle.c
Modules/_queuemodule.c
Modules/_randommodule.c
Modules/_sqlite/module.h
Modules/_ssl.c
Modules/_ssl.h
Modules/_struct.c
Modules/_testmultiphase.c
Modules/_threadmodule.c
Modules/_xxinterpchannelsmodule.c
Modules/_zoneinfo.c
Modules/arraymodule.c
Modules/cjkcodecs/multibytecodec.c
Modules/clinic/_testmultiphase.c.h
Modules/itertoolsmodule.c
Modules/socketmodule.c
Python/Python-tokenize.c

Linked PRs

  • gh-116937

vstinner avatar Mar 17 '24 21:03 vstinner

Attached PR gh-116937 implements the change.

vstinner avatar Mar 17 '24 21:03 vstinner

The function was added by issue gh-90771.

vstinner avatar Mar 18 '24 11:03 vstinner

I agree. It can be implemented using only limited API, but the CPython implementation can pull tricks that make it worth calling as a function.

I'll be interested in what other members of the C-API WG have to say.

encukou avatar Mar 19 '24 08:03 encukou

It can be implemented using only limited API

I don't think that it's possible in practice. For example, Python 3.13 adds BEGIN_TYPE_LOCK() with _PyCriticalSection which is not part of the stable ABI.

vstinner avatar Mar 19 '24 09:03 vstinner

AFAIK, that's for the freethreading experiment. I hope that when the experiment is done, it won't require adding non-limited calls to stable ABI extensions.

encukou avatar Mar 20 '24 12:03 encukou

@encukou:

I'll be interested in what other members of the C-API WG have to say.

Ok, I created issue https://github.com/capi-workgroup/decisions/issues/20.

vstinner avatar Mar 21 '24 10:03 vstinner

Ok, I created issue https://github.com/capi-workgroup/decisions/issues/20.

The C API Working Group approved this addition: https://github.com/capi-workgroup/decisions/issues/20.

vstinner avatar Mar 25 '24 16:03 vstinner

Implemented in change https://github.com/python/cpython/commit/507896d97dcff2d7999efa264b29d9003c525c49

vstinner avatar Mar 25 '24 16:03 vstinner