AcademySoftwareFoundation
[REQUEST] Replace Boost.Python with PyBind11
Looking for help with Boost.Python -> PyBind11 migration
These points represent the work required to modify python bindings, located in openvdb/python/, to build with PyBind11 and remove the boost-python dependency. Any work on the below should be done off of the feature/pybind branch. This branch will be synced with master on releases and allows for any work to be contributed incrementally:
- Completely replace the
boost-pythondependency for the pyopenvdb module withPyBind11[*]. This includes working with NumPy support. - Maintain API and feature parity with the existing
boost-pythonbindings. The Python unit tests should ideally require no modification and pass as they are after this work has been completed. - Maintain the existing folder structure/file layout where possible
- Use the existing CMake build system and work with OpenVDBs minimum version requirements, including CMake, GCC/Clang, CXX standard, Python versions >= 2.7 and
PyBind11versions >= 2.6 [**]. - Conform to the OpenVDB Coding standards
- Have commits signed-off (DCO checks) and contributors should be prepared to sign a CLA.
- Bonus: Modify the CI, where necessary, to build and test with PyBind11. See
.github/workflows/build.yml
[*] Note that, similar to the existing build system, PyBind11 should be expected to exist in a users environment and should not be incorporated into the OpenVDB repository (there are outstanding licensing questions with this approach so this should not be done at this stage).
[**] PyBind11 2.6 was released in October 2020. The latest version is 2.6.2. I think ensuring it works with 2.6.X is enough.
Started on an attempt for this (uses the scikit-build system) but am inexperienced with both pybind11 and openvdb... so any collaboration is welcome. https://github.com/chrisflesher/openvdb/tree/replace-boost-python-with-pybind11
As a reference, I have been working with OpenVDB and pybind11 for 1 year now and I have already some open-source bindings:
I don't have the time to carry on this effort right now (although I'd love to get rid of boost python!) But I hope this helps to whoever is/will be working on this issue
If someone knows a lot about C++ templates I could use some help.
Am trying to port the custom type converters from pyboost to pybind11 and successfully wrote one for Vec2i. However there are many of these (e.g. Vec2I, Vec2s, Vec2d, Vec3i, ...) and am having trouble figuring out how to do this for a custom type_caster.
How should I generalize openvdb::Vec2i to typename VecT in the following example?
template <> struct type_caster<openvdb::Vec2i> {
public:
PYBIND11_TYPE_CASTER(openvdb::Vec2i, const_name("Vec2i"));
bool load(handle src, bool) {
// python -> C++ code goes here
}
static handle cast(openvdb::Vec2i src, return_value_policy, handle) {
// C++ -> python code goes here
}
};
How should I generalize
openvdb::Vec2itotypename VecTin the following example?
I would actually work with preprocessor macros here.
Refer to
- https://github.com/mitsuba-renderer/drjit/blob/master/src/python/bind_llvm.cpp
- https://github.com/mitsuba-renderer/drjit/blob/master/src/python/bind.h
for some examples of complex templated binding code. They work with a combination of templated binding functions (top pf bind.h) and preprocessor macros (bottom of bind.h) to generate the binding code.
I've been looking into pybind11 for OpenVDB. I have all of the Python bindings ported over from boost::python to pybind11 and all of the Python unit tests are passing (without modification). There are still a few issues here and there that I need to address, but it's looking promising.
I've been looking into pybind11 for OpenVDB. I have all of the Python bindings ported over from boost::python to pybind11 and all of the Python unit tests are passing (without modification). There are still a few issues here and there that I need to address, but it's looking promising.
I'm very happy to hear that, this has been on our roadmap for a long time. Thanks for the update Matt and feel free to join one of our upcoming weekly TSC meetings if you want to discuss / present anything. :)
