[FEA]: change function __qualname__ prefix

[gentlegiantJGC]

Required prerequisites

• [x] Make sure you've read the documentation. Your issue may be addressed there.
• [x] Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• [x] Consider asking first in the Gitter chat room or in a Discussion.

What version (or hash if on master) of pybind11 are you using?

3.0.0

Problem description

When defining a function with pybind11 and inspecting the __qualname__ attribute in python it has a prefix that I was not expecting. In Pybind11 2.13.6 __qualname__ was PyCapsule.function_name In Pybind11 3.0.0 __qualname__ is pybind11_detail_function_record_v1_msvc_md_mscver19.function_name

Is this expected behaviour?

I was expecting this value to just be function_name

I noticed this when running pybind11 stubgen on the library built with the new version of pybind11 and it created an import like the following. This can probably be patched in pybind11 stubgen if this behaviour is intended.

from mod.pybind11_detail_function_record_v1_msvc_md_mscver19 import function_name

PEP 3155 defines __qualname__

Reproducible example code

#include <pybind11/pybind11.h>

PYBIND11_MODULE(mod, m)
{
    m.def("function_name", [](){});
}

Is this a regression? Put the last known working version here if it is.

Not a regression

[gentlegiantJGC]

#2059 describes the same issue for methods

[rwgk]

In Pybind11 2.13.6 __qualname__ was PyCapsule.function_name In Pybind11 3.0.0 __qualname__ is pybind11_detail_function_record_v1_msvc_md_mscver19.function_name

Is this expected behaviour?

Good question, TBH, I don't know and never actually thought about it.

Did you already inspect the current implementation, to see how we could achieve the __qualname__ you're expecting?

See also: PR #5771 (changes include/pybind11/detail/function_record_pyobject.h, where you'd likely have to make changes, too)

[gentlegiantJGC]

The __qualname__ output is created in this function in cpython

It pulls the prefix from the m_self attribute. If m_self is NULL or a module type it does not add a prefix.

Pybind always sets this attribute as a pointer to the internal function record type which makes python believe the function is bound to that instance and thus it adds the instance prefix.

The only solution I can see is to correctly set m_self to the module or class object the function or method is defined on so that the cpython function creates it correctly.

I don't where else pybind uses m_self so this may not be a trivial change.

#2059 is the same issue for methods but I believe the root issue is the same. This post from that thread goes into the issue in more detail.

[rwgk]

I spent some time exploring possible fixes for the long qualname problem in pybind11 3.x. My experiments are collected in this branch: https://github.com/rwgk/pybind11/tree/function_record_qualname

In case anyone wants to continue the work later, here is a summary of what I tried and what we learned — so future experiments can avoid retracing the same paths:

✅ Confirmed behavior qualname for pybind11 functions is derived by CPython’s meth_get__qualname__ using m_self and its type’s qualname.

pybind11 currently sets m_self to the function_record_PyObject instance, which has a long internal name (pybind11_detail_function_record_*).

❌ Things we tried that did not work m_self = nullptr to force short names

Qualname is short ✅

But overload chaining and lifetime broke because pybind11 relies on m_self.

Use m_module tuple for lifetime ((module, function_record))

Works for lifetime ✅

Breaks module because CPython exposes m_module directly.

Assigning module via setattr overwrites m_module, losing our tuple → breaks lifetime.

Hidden attribute like pybind11_function_record

Fails because PyCFunctionObject (builtin functions/methods) has no dict → cannot store arbitrary attributes.

Type spoofing (PyModule_Type or tp_bases hack)

Goal: make PyModule_Check(m_self) true so meth_get__qualname__ returns short name.

Caused segfaults and GC corruption because function_record_PyObject layout is incompatible with PyModule_Type.

Manual qualname assignment

PyCFunctionObject.qualname is read‑only → raises AttributeError.

tp_bases / tp_mro post‑creation hacks

Attempting to fake module inheritance after type creation led to memory corruption and xdist chaos.

⚠️ Observations PyCFunctionObject is extremely constrained:

module is just m_module

qualname is read‑only and computed on the fly

No dict → no arbitrary attributes

There is no “extra slot” for storing the function record if m_self = NULL.

📌 Potential future directions Proxy object as m_self

A lightweight heap type with a short qualname (e.g. "" or "pybind11").

Holds a reference to the real function_record internally.

CPython sees the proxy → short qualname, pybind11 uses proxy to reach the real record.

Global C++ map for lifetime

Keep m_self = NULL for short qualname

Keep m_module as the string for correct module

Store m_ptr → py_func_rec in a static map for lifetime and overload resolution.

At this point I decided to pause the effort. The branch above contains all my experiments, including the ones that produced segfaults, so that anyone who wants to continue can start from there.

[gentlegiantJGC]

I am getting chatGPT vibes from this message but I trust that you actually researched it. I thought it was going to be a difficult issue to solve.

[rwgk]

I am getting chatGPT vibes from this message but I trust that you actually researched it.

I was mostly following ChatGPT suggestions, but what you see on the https://github.com/rwgk/pybind11/tree/function_record_qualname branch is all real of course.

Essentially, the combination of my own ideas and ChatGPT suggestions all resulted in dead ends :-(

I spent 4+ hours on the exploring the dead ends.

Not sure when I'll have time to poke around more.

[rwgk]

@henryiii @gentlegiantJGC Turning this over in the back of my mind, I got to think using inheritance from https://docs.python.org/3/c-api/structures.html#c.PyCFunction might be worth trying. That's something I haven't explored yet.

For anyone wanting to work on this: I recommend starting with https://github.com/rwgk/pybind11/tree/function_record_qualname, or equivalently, with this simple diff:

diff --git a/include/pybind11/detail/function_record_pyobject.h b/include/pybind11/detail/function_record_pyobject.h
index d9c5bad9..6f2a62ad 100644
--- a/include/pybind11/detail/function_record_pyobject.h
+++ b/include/pybind11/detail/function_record_pyobject.h
@@ -187,5 +187,16 @@ inline PyObject *reduce_ex_impl(PyObject *self, PyObject *, PyObject *) {
 
 PYBIND11_NAMESPACE_END(function_record_PyTypeObject_methods)
 
+inline PyObject *extract_function_record(PyObject *func_obj) {
+    PyObject *self = PyCFunction_GET_SELF(func_obj);
+    if (!self) {
+        set_error(PyExc_RuntimeError,
+                  str("pybind11::detail::extract_function_record(") + repr(func_obj)
+                      + str(") FAILED"));
+        return nullptr;
+    }
+    return self;
+}
+
 PYBIND11_NAMESPACE_END(detail)
 PYBIND11_NAMESPACE_END(PYBIND11_NAMESPACE)
diff --git a/include/pybind11/functional.h b/include/pybind11/functional.h
index 8f59f5fe..eda1b291 100644
--- a/include/pybind11/functional.h
+++ b/include/pybind11/functional.h
@@ -88,7 +88,7 @@ public:
            captured variables), in which case the roundtrip can be avoided.
          */
         if (auto cfunc = func.cpp_function()) {
-            auto *cfunc_self = PyCFunction_GET_SELF(cfunc.ptr());
+            auto *cfunc_self = extract_function_record(cfunc.ptr());
             if (cfunc_self == nullptr) {
                 PyErr_Clear();
             } else {
diff --git a/include/pybind11/pybind11.h b/include/pybind11/pybind11.h
index 76519ad2..de599076 100644
--- a/include/pybind11/pybind11.h
+++ b/include/pybind11/pybind11.h
@@ -602,10 +602,10 @@ protected:
         detail::function_record *chain = nullptr, *chain_start = rec;
         if (rec->sibling) {
             if (PyCFunction_Check(rec->sibling.ptr())) {
-                auto *self = PyCFunction_GET_SELF(rec->sibling.ptr());
+                auto *self = detail::extract_function_record(rec->sibling.ptr());
                 if (self == nullptr) {
-                    pybind11_fail(
-                        "initialize_generic: Unexpected nullptr from PyCFunction_GET_SELF");
+                    pybind11_fail("initialize_generic: Unexpected nullptr from "
+                                  "detail::extract_function_record");
                 }
                 chain = detail::function_record_ptr_from_PyObject(self);
                 if (chain && !chain->scope.is(rec->scope)) {
@@ -667,7 +667,7 @@ protected:
                 chain_start = rec;
                 rec->next = chain;
                 auto *py_func_rec
-                    = (detail::function_record_PyObject *) PyCFunction_GET_SELF(m_ptr);
+                    = (detail::function_record_PyObject *) detail::extract_function_record(m_ptr);
                 py_func_rec->cpp_func_rec = unique_rec.release();
                 guarded_strdup.release();
             } else {
@@ -2568,7 +2568,7 @@ private:
             return nullptr;
         }
 
-        handle func_self = PyCFunction_GET_SELF(h.ptr());
+        handle func_self = detail::extract_function_record(h.ptr());
         if (!func_self) {
             throw error_already_set();
         }

From there:

Modify this

https://github.com/pybind/pybind11/blob/a5665e3acacc7251104d5f2ee7fdba9b5293ba8e/include/pybind11/pybind11.h#L634-L643

and the that new extract_function_record() in tandem Until It Works (tm).

[rwgk]

I worked on this some more on a 12 hour flight back home, using ChatGPT 5 Pro a lot, which means it's often thinking for minutes before it produces a response. The full conversation is here:

https://chatgpt.com/share/68a18713-5940-8008-b52c-de3137325f00

It's probably impossible to follow in all details, but the gist is summarized near the (current) end: look for

compact “parking‑lot” checklist

TL;DR: It's ridiculously hard to get the qualname we want. The most promising avenue seems to be to implement a wrapper type. I got pretty far with that (pycfunctionobject_wrapper.h), but time is up for now. What I have at the moment is in this branch: https://github.com/rwgk/pybind11/tree/function_record_qualname

[gentlegiantJGC]

Currently dispatcher is a stateless static function and you use self to pass it the state to recover the function record. Could dispatcher be a lambda which captures the function record? That would allow self to be what python requires. You would need to store the lambda somewhere and pass the raw pointer to cpython.

Edit: looking into this more I don't think a lambda can be converted to a C function pointer. I assumed it would be possible.

[rwgk]

Currently dispatcher is a stateless static function and you use self to pass it the state to recover the function record. Could dispatcher be a lambda which captures the function record? That would allow self to be what python requires. You would need to store the lambda somewhere and pass the raw pointer to cpython.

Edit: looking into this more I don't think a lambda can be converted to a C function pointer. I assumed it would be possible.

I copy-pasted the above into my existing ChatGPT conversation, then updated the link:

https://chatgpt.com/share/68a18713-5940-8008-b52c-de3137325f00

Search for:

Short answer: Using a lambda to “carry”

TL;DR: It looks like a no-go.

My own state of mind at this minute:

• General: pybind11 functions never had a meaningful qualname, it just changed as a consequence of PR #5580.
• Upstream a CPython tweak: have meth_get__qualname__ honor an optional __objclass__ on m_self. Then pybind11 can keep using m_self for dispatch and get a short qualname with zero wrapper overhead.
• Slowly (a few years) but nicely pybind11 functions will gain meaningful qualnames.

[rwgk]

This is a first completely untested approximation to what the small (almost tiny) cpython change would look like:

https://github.com/rwgk/cpython/commit/52a03ff27415d028c6982ac5e25958e43c8eb8e6

Anyone with cpython development experience seeing this: It'd be awesome if someone could help/take over working on this.

[ngoldbaum]

Anyone with cpython development experience seeing this: It'd be awesome if someone could help/take over working on this.

I think because you're adding a new dunder method, this probably would need a PEP?

The first step for this sort of thing is to bring it to the #ideas category on discuss.python.org.

[rwgk]

The first step for this sort of thing is to bring it to the #ideas category on discuss.python.org.

Active help would be highly appreciated.

I changed the title of this issue to [FEA] because this isn't really a bug, it's just that one non-sensical name was replaced with another one, as a side-effect of enabling a new feature. Achieving a meaningful qualname is another feature.