__qualname__ for methods

[m-chaturvedi]

The __qualname__ attribute for methods created using pybind11 seems to be not consistent with that created by cpython. Testing Repo: https://github.com/m-chaturvedi/reproduce_issues/tree/master/pybind_qualname Related to #1171 via @jagerman . PEP: https://www.python.org/dev/peps/pep-3155/

$  /bin/cat main.py
#!/usr/bin/python3
# Tested with python 3.7

import python_module
import pybind_module
from pprint import pprint

if __name__ == "__main__":
    assert python_module.Pet.get_data.__qualname__ == "Pet.get_data"
    assert python_module.Pet.__init__.__qualname__ == "Pet.__init__"

    assert pybind_module.Pet.get_data.__qualname__ == "PyCapsule.get_data"
    assert pybind_module.Pet.__init__.__qualname__ == "PyCapsule.__init__"

$  /bin/cat main.cpp
#include <pybind11/pybind11.h>
#include <iostream>
namespace py = pybind11;

class Pet {
 public:
        Pet() {
            c = 10;
        }
        ~Pet() {}
        int get_data() {
            return c;
        }

 private:
        int c;
};


PYBIND11_MODULE(pybind_module, m) {
    using rvp = pybind11::return_value_policy;
    py::class_<Pet>(m, "Pet")
        .def(py::init<>())
        .def("get_data", &Pet::get_data);
}

$  /bin/cat python_module.py

class Pet(object):

    def __init__(self):
        pass

    def get_data(self):
        return 10

$  /bin/cat CMakeLists.txt
cmake_minimum_required(VERSION 2.8.5)
project(test_pybind11 CXX)

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
# Tested with python 3.7
set(PATH_TO_PYBIND /home/chaturvedi/workspace/pybind11/build_master_ninja/install/share/cmake/pybind11)
# IMPORT pybind11
find_package(pybind11 REQUIRED CONFIG PATHS ${PATH_TO_PYBIND})

configure_file(main.py ${CMAKE_BINARY_DIR} COPYONLY)
configure_file(python_module.py ${CMAKE_BINARY_DIR} COPYONLY)
pybind11_add_module(pybind_module main.cpp)

[Skylion007]

Also could be due to self pointing to an unbound PyBind11 class and therefore not being converted properly.

[Skylion007]

May be related to: https://github.com/pybind/pybind11/issues/2722

[XuehaiPan]

The logic for PyCFunctionObject.__qualname__ goes here:

https://github.com/python/cpython/blob/f84754b70506f03bcbf9fb0265e327d05a1a4b51/Objects/methodobject.c#L212-L244

    /* If __self__ is a module or NULL, return m.__name__
       (e.g. len.__qualname__ == 'len')

       If __self__ is a type, return m.__self__.__qualname__ + '.' + m.__name__
       (e.g. dict.fromkeys.__qualname__ == 'dict.fromkeys')

       Otherwise return type(m.__self__).__qualname__ + '.' + m.__name__
       (e.g. [].append.__qualname__ == 'list.append') */

In pybind11, we set fn->m_ml->def->ml_meth = py::cpp_function::dispatcher: L547-L548

https://github.com/pybind/pybind11/blob/01169061891d8f5609d8cdf10f6dc396ef4ccd44/include/pybind11/pybind11.h#L544-L549

and set fn->m_self = rec_capsule which is a py::capsule object: L551-L553, L565

https://github.com/pybind/pybind11/blob/01169061891d8f5609d8cdf10f6dc396ef4ccd44/include/pybind11/pybind11.h#L551-L565

We use the fn->m_self record capsule in the py::cpp_function::dispatcher function as the self argument to support overloading:

https://github.com/pybind/pybind11/blob/01169061891d8f5609d8cdf10f6dc396ef4ccd44/include/pybind11/pybind11.h#L717

---

In best practice, the m_self field should be nullptr, a module object, or a type object. However, we have a PyCapsule object (the rec_capsule) in pybind11.

Then in https://github.com/python/cpython/blob/f84754b70506f03bcbf9fb0265e327d05a1a4b51/Objects/methodobject.c#L225-L230, we get:

type_qualname = Py_TYPE(fn->m_self).tp_name  // "PyCapsule"

this results in the following:

fn.__qualname__  # -> f'PyCapsule.{fn.__name__}'

[XuehaiPan]

I think the only way is to use currying the function_record in the dispatcher and make __self__ to be NULL in the PyCFunctionObject. Any thoughts?

[gentlegiantJGC]

In best practice, the m_self field should be nullptr, a module object, or a type object. However, we have a PyCapsule object (the rec_capsule) in pybind11.

Based on the CPython implementation self should be:

1. The owning module for functions defined in the root of the module
2. The owning class type for unbound methods
3. The owning instance for bound methods

Pybind11 seems to be using this field incorrectly.