Implement PEP 793 – PyModExport

[encukou]

Feature or enhancement

Proposal:

See PEP 793. I'll copy the abstract for reference:

In this PEP, we propose a new entry point for C extension modules, by which one can define a module using an array of PyModuleDef_Slot structures without an enclosing PyModuleDef structure. This allows extension authors to avoid using a statically allocated PyObject, lifting the most common obstacle to making one compiled library file usable with both regular and free-threaded builds of CPython.

To make this viable, we also specify new module slot types to replace PyModuleDef’s fields, and to allow adding a token similar to the Py_tp_token used for type objects.

We also add an API for defining modules from slots dynamically.

The existing API (PyInit_*) is soft-deprecated. (That is: it will continue to work without warnings, and it’ll be fully documented and supported, but we plan to not add any new features to it.)

---

• [x] Initial implementation (#140556)
• [ ] Docs:
  • [x] Reference docs (#141197)
    • [x] Soft-deprecate PyInit_* (#141197)
    • [x] Better explanation of module state (#141197)
    • [x] Give an example for tokens (#141197)
  • [ ] What's New
    • [ ] a notice for DYNLOADFILE users
  • [ ] Porting guide HOWTO
    • [ ] Document that the export hook runs without GIL in FT builds (see #141780)
  • [ ] Tutorial update
  • [ ] Mentions in PyImport_AppendInittab & friends
  • [ ] Make sure references to new API are linked:
    • [x] PyMod_Execin PyImport_CreateModuleFromInitfunc docs is linked (#141197)
    • [ ] PyMODEXPORT_FUNC in blurb for #141672
    • [ ] blurb for #141780
• [ ] xxlimited.c update (#142827)
  • [ ] Remove/replace _Py_OPAQUE_PYOBJECT in xxlimited.c (by 3.15.0 beta)

Optional stretch goals:

• [ ] Remove the PyInit function from Lib/test/test_cext/extension.c (https://github.com/python/cpython/pull/141511)
• [ ] Speed & size improvements
  • [ ] importdl: load the DLL once
  • [x] bool for the GIL flag (#141519)
• [ ] No longer require spec.name in PyModule_FromSlotsAndSpec
• [ ] No longer require static allocation for Py_mod_methods
• [ ] Add a PyModule_GetStateWithToken() function

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

• gh-140556
• gh-141056
• gh-141197
• gh-141511
• gh-141519
• gh-142583
• gh-142827

See also:

• gh-141672

[encukou]

I'd appreciate a review of the reference docs update: #141197