pybind11
pybind11 copied to clipboard
[FEATURE REQUEST]: Dynamic class names (`pybind11::class_` constructor taking `std::string`)
Required prerequisites
- [X] Make sure you've read the documentation. Your issue may be addressed there.
- [X] Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
- [X] Consider asking first in the Gitter chat room or in a Discussion.
What version (or hash if on master) of pybind11 are you using?
2.10.4
Problem description
I often find myself wrapping C++ templates using pybind11. In this case, I can write a template function that instantiates and registers a specific form of the template. Of course, if I do this for multiple instantiations, I end up with multiple python classes, and those should have distinct names. I would like to generate these names in the template function at runtime. The problem is that the class_ constructors only take const char * for the name, providing no way of managing the dynamic memory where the runtime-generated name resides (a naive use of std::string::c_str() obviously leads to dangling references).
Python's type system probably wasn't designed with C++ templates in mind, so it likely doesn't provide an direct way of resource management for the name of a type object. However, we can make a workaround using weak references, following the example given in the pybind11 documentation. I have been using the following:
template <typename Cls, typename... Options, typename... Args>
auto make_class(py::module_ &m, std::string name, Args &&...args) {
auto holder = std::make_unique<std::string>(std::move(name));
auto ret = py::class_<Cls, Options...>(m, holder->c_str(), std::forward<Args>(args)...);
py::cpp_function cleanup_callback([holder = std::move(holder)](py::handle weakref) mutable {
holder.reset();
weakref.dec_ref();
});
(void)pybind11::weakref(ret, cleanup_callback).release();
return ret;
}
I wonder if something like this could be reasonably included in a dedicated constructor?
Reproducible example code
No response
Is this a regression? Put the last known working version here if it is.
Not a regression
#include <pybind11/pybind11.h>
#include <pybind11/functional.h>
#include <memory>
namespace py = pybind11;
// Helper function to create a py::class_ with a dynamic class name
template <typename Cls, typename... Options, typename... Args>
auto make_class(py::module_ &m, std::string name, Args&&... args) {
// Use a unique_ptr to manage the lifetime of the dynamically generated name
auto holder = std::make_unique<std::string>(std::move(name));
// Create the py::class_ object, using holder->c_str() for the name
auto ret = py::class_<Cls, Options...>(m, holder->c_str(), std::forward<Args>(args)...);
// Create a cleanup callback that will be invoked when the weakref is garbage collected
py::cpp_function cleanup_callback([holder = std::move(holder)](py::handle weakref) mutable {
holder.reset(); // Release the string memory
weakref.dec_ref(); // Decrease the reference count for the weakref
});
// Create a weak reference for the class to ensure proper cleanup of the dynamically allocated string
(void)pybind11::weakref(ret, cleanup_callback).release();
return ret;
}
class MyClass {
public:
MyClass(int x) : x(x) {}
int x;
};
PYBIND11_MODULE(example, m) {
// Use the make_class helper function to dynamically generate a class name at runtime
make_class<MyClass>(m, "DynamicClassName", py::init<int>());
}
`#include <pybind11/pybind11.h>
#include <pybind11/functional.h>
#include <memory>
namespace py = pybind11;
// Helper function to create a py::class_ with a dynamic class name
template <typename Cls, typename... Options, typename... Args>
auto make_class(py::module_ &m, std::string name, Args&&... args) {
// Use a unique_ptr to manage the lifetime of the dynamically generated name
auto holder = std::make_unique<std::string>(std::move(name));
// Create the py::class_ object, using holder->c_str() for the name
auto ret = py::class_<Cls, Options...>(m, holder->c_str(), std::forward<Args>(args)...);
// Create a cleanup callback that will be invoked when the weakref is garbage collected
py::cpp_function cleanup_callback([holder = std::move(holder)](py::handle weakref) mutable {
holder.reset(); // Release the string memory
weakref.dec_ref(); // Decrease the reference count for the weakref
});
// Create a weak reference for the class to ensure proper cleanup of the dynamically allocated string
(void)pybind11::weakref(ret, cleanup_callback).release();
return ret;
}
class MyClass {
public:
MyClass(int x) : x(x) {}
int x;
};
PYBIND11_MODULE(example, m) {
// Use the make_class helper function to dynamically generate a class name at runtime
make_class<MyClass>(m, "DynamicClassName", py::init<int>());
}
#include <pybind11/pybind11.h>
#include <pybind11/functional.h>
#include <memory>
namespace py = pybind11;
// Helper function to create a py::class_ with a dynamic class name
template <typename Cls, typename... Options, typename... Args>
auto make_class(py::module_ &m, std::string name, Args&&... args) {
// Use a unique_ptr to manage the lifetime of the dynamically generated name
auto holder = std::make_unique<std::string>(std::move(name));
// Create the py::class_ object, using holder->c_str() for the name
auto ret = py::class_<Cls, Options...>(m, holder->c_str(), std::forward<Args>(args)...);
// Create a cleanup callback that will be invoked when the weakref is garbage collected
py::cpp_function cleanup_callback([holder = std::move(holder)](py::handle weakref) mutable {
holder.reset(); // Release the string memory
weakref.dec_ref(); // Decrease the reference count for the weakref
});
// Create a weak reference for the class to ensure proper cleanup of the dynamically allocated string
(void)pybind11::weakref(ret, cleanup_callback).release();
return ret;
}
class MyClass {
public:
MyClass(int x) : x(x) {}
int x;
};
PYBIND11_MODULE(example, m) {
// Use the make_class helper function to dynamically generate a class name at runtime
make_class<MyClass>(m, "DynamicClassName", py::init<int>());
}
@ljluestc: What exactly are you trying to say with the code-examples you provided? IMHO, it looks like you just copied the implementation I provided above, added a few comments, and made a full working example out of it (also, it looks like there is a tripple copy-paste in there). As it stands, that doesn't add anything to the discussion but only distracts from it with a large blob of code... I'd appreciate if you could correct the code example, and add a few words on why you post that code.
My intention was to discuss if it were reasonable to add a corresponding constructor to py::class_ instead, which would take a std::string, and basically did all of my make_class internally as needed.