pybind11
pybind11 copied to clipboard
[BUG]: tp_traverse occasionally executed before C++ ctor
Required prerequisites
- [X] Make sure you've read the documentation. Your issue may be addressed there.
- [X] Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
- [ ] Consider asking first in the Gitter chat room or in a Discussion.
What version (or hash if on master) of pybind11 are you using?
2.11.1
Problem description
Enabling Py_TPFLAGS_HAVE_GC in custom_type_setup will occasionally call type->tp_traverse before the C++ ctor. Depending on what the C++ object contains, this can be anything from harmless (if the allocation is zero filled and traversing it just visits some nulls), to segfault (if all-zeroes is not a legal state for the object, for example trying to iterate a libstdc++ std::unordered_map), or worse (if the allocation is not zero filled, the worst case is a security hole).
I don't know if this is a bug in pybind11, Python itself (calling _PyObject_GC_TRACK in PyType_GenericAlloc in Objects/typeobject.c seems kinda suspicious to me), or in my code, but if it's the latter, the same bug also exists at https://pybind11.readthedocs.io/en/stable/advanced/classes.html#custom-type-setup .
Reproducible example code
#include <pybind11/pybind11.h>
#include <unordered_set>
namespace py = pybind11;
class funny_class;
std::unordered_set<funny_class*> items_that_exist;
class funny_class {
public:
funny_class() { items_that_exist.insert(this); }
~funny_class() { items_that_exist.erase(this); }
};
PYBIND11_MODULE(example, m) {
py::class_<funny_class> elem_container(m, "fun",
py::custom_type_setup([](PyHeapTypeObject* heap_type) {
PyTypeObject* type = &heap_type->ht_type;
type->tp_flags |= Py_TPFLAGS_HAVE_GC;
type->tp_traverse = [](PyObject* self_base, visitproc visit, void* arg) {
auto& self = py::cast<funny_class&>(py::handle(self_base));
if (!items_that_exist.count(&self))
puts("ERROR: Item was traversed without being constructed.");
return 0;
};
type->tp_clear = [](PyObject* self_base) {
auto& self = py::cast<funny_class&>(py::handle(self_base));
if (!items_that_exist.count(&self))
puts("ERROR: Item was erased without being constructed.");
return 0;
};
}));
elem_container.def(py::init<>());
}
import example
a = [(example.fun(), example.fun())[1] for _ in range(10000)]
Is this a regression? Put the last known working version here if it is.
Not a regression