[BUG]: throw is not propagate from python in some circumstances

[alkino]

Required prerequisites

• [X] Make sure you've read the documentation. Your issue may be addressed there.
• [X] Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• [X] Consider asking first in the Gitter chat room or in a Discussion.

Problem description

Here is a code that throw sometimes but not other time. c.go() throw, call_go(c) that call c.go() does not throw. The call_go(c) go through https://github.com/pybind/pybind11/blob/master/include/pybind11/pytypes.h#L753-L759 because of https://github.com/pybind/pybind11/blob/master/include/pybind11/pybind11.h#L2659

Do you know if it is a "bug" or a normal behavior that I don't understand.

Regards

Reproducible example code

// C++ file
#include <string>
#include <pybind11/pybind11.h>
 
namespace py = pybind11;
class Animal {
public:
    virtual ~Animal() { }
    virtual std::string go() { return "aaa"; }
};
 
class PyAnimal : public Animal {
public:
    using Animal::Animal;
 
    std::string go() override {
        PYBIND11_OVERRIDE(std::string, Animal, go, );
    }
};
 
std::string call_go(Animal* animal) {
    return animal->go();
}
 
PYBIND11_MODULE(foo, m) {
    py::class_<Animal, PyAnimal>(m, "Animal")
        .def(py::init<>())
        .def("go", &Animal::go);
    m.def("call_go", &call_go);
}
 
// Python file
from foo import *
 
class Cat(Animal):
    def __getattribute__(self, _):
        print("*** raising an exception now ***")
        raise Exception("uh oh")
 
c = Cat()
# c.go() // throw
call_go(c) // no throw

[alkino]

On gitter, Mikaël Capelle told me:

Yes, indeed the exception is cleared in Pybind11... Unfortunately, there is probably no way around that since it's most likely impossible to know if PyObject_GetAttrString returns NULL because the attribute was missing or because an exception was thrown. Basically, you call getattr(c, "go"), but via PyObject_GetAttrString. If the call returns a null-pointer (error, or simply because the attribute is missing), then you call to the base-class function.

When you do animal->go(), this goes through the PYBIND11_OVERRIDE macro, which will call getattribute through PyObject_GetAttrString. Using the C-API, there is no throw, a simple flag is set to indicate that an error was thrown and PyObject_GetAttrString returns a null-pointer. In this case, pybind11 assumes that the null-pointer is because Cat does not implement go and thus falls back to Animal::go. There is no easy way to distinguish between PyObject_GetAttrString returning a null-pointer because Cat does not implement go or because getattribute throws or returns None.

[Skylion007]

@alkino This could be fixed actually but doing an explicit PyErr_Ocurred check on our end to see if an Exception is thrown in getattribute. It's not clear how we would detect if it returned None though.