Linking Python Bindings Code

[peterdavidfagan]

trafficstars

Hi,

I'm a PhD student and I'm trying to integrate the MuJoCo interactive viewer with an integration of mujoco and ROS. To accomplish this I intended to pass pointers for a simulation model and data from C++ to a passive viewer instance in Python.

I'm looking for some help with linking against the structs module implementation of MjModelWrapper and MjDataWrapper such that I can return types in my Python bindings that are compatible with the MuJoCo Python library. My get methods are quite simple and so far look as follows:

mujoco::python::_impl::MjModelWrapper FrankaMJROS::get_model()
{
  return *mujoco::python::_impl::MjModelWrapper::FromRawPointer(this->m);
}

I've attempted to manually build a static library for structs and to link against this in my bindings library. I am finding the process of building the python binding code to be quite involved as the build procedure involves copying multiple files while also configuring CMake arguments most of which happens in bash and a python build extension. Thus far I have been able to start a regular CMake build where right now I manually configure and copy files (only for testing purposes). This being said I encounter build errors relating to the generated header for enum_traits.h (and I suspect I will find the same for function_traits.h):

In file included from /home/peter/Code/temp/ros2_robotics_research_toolkit/src/mujoco/mujoco_ros/python_type_caster/mujoco/python/mujoco/enums.cc:22:
/home/peter/Code/temp/ros2_robotics_research_toolkit/src/mujoco/mujoco_ros/python_type_caster/mujoco/python/mujoco/enum_traits.h:253:45: error: ‘mjOBJ_FRAME’ is not a member of ‘mjtObj’ {aka ‘mjtObj_’}
  253 |     std::make_pair("mjOBJ_FRAME", ::mjtObj::mjOBJ_FRAME)};
      |                                             ^~~~~~~~~~~
/home/peter/Code/temp/ros2_robotics_research_toolkit/src/mujoco/mujoco_ros/python_type_caster/mujoco/python/mujoco/enum_traits.h:322:52: error: ‘mjSENS_GEOMDIST’ is not a member of ‘mjtSensor’ {aka ‘mjtSensor_’}
  322 |     std::make_pair("mjSENS_GEOMDIST", ::mjtSensor::mjSENS_GEOMDIST),
      |                                                    ^~~~~~~~~~~~~~~
/home/peter/Code/temp/ros2_robotics_research_toolkit/src/mujoco/mujoco_ros/python_type_caster/mujoco/python/mujoco/enum_traits.h:323:54: error: ‘mjSENS_GEOMNORMAL’ is not a member of ‘mjtSensor’ {aka ‘mjtSensor_’}
  323 |     std::make_pair("mjSENS_GEOMNORMAL", ::mjtSensor::mjSENS_GEOMNORMAL),
      |                                                      ^~~~~~~~~~~~~~~~~
/home/peter/Code/temp/ros2_robotics_research_toolkit/src/mujoco/mujoco_ros/python_type_caster/mujoco/python/mujoco/enum_traits.h:324:54: error: ‘mjSENS_GEOMFROMTO’ is not a member of ‘mjtSensor’ {aka ‘mjtSensor_’}
  324 |     std::make_pair("mjSENS_GEOMFROMTO", ::mjtSensor::mjSENS_GEOMFROMTO),
      |                                                      ^~~~~~~~~~~~~~~~~
^Cmake[2]: *** [CMakeFiles/_enums.dir/build.make:76: CMakeFiles/_enums.dir/enums.cc.o] Interrupt
make[1]: *** [CMakeFiles/Makefile2:977: CMakeFiles/_enums.dir/all] Interrupt
make: *** [Makefile:136: all] Interrupt

I have configured cmake to point towards the correct cmake folder and mujoco library/include directories.

I have posted on discord to see if others have tried to accomplish the same thing, it seems one other person has but I am waiting for their response in case they have an example. Right now I am considering if there is anything within the pybind11 API that would enable me to cast from the raw pointer type for a given struct to the return type required for the binded Python library.

Any advice on linking against the bindings code in an appropriate manner is appreciated. I will update this thread if I come to a solution myself. Manually configuring the build takes a bit of work but I suspect it might be the only way to incorporate the Python binding wrappers into my code.

[peterdavidfagan]

Browsing GitHub issues some more it seems there are already related issues that are closed, adding these for reference:

• https://github.com/google-deepmind/mujoco/issues/983
• https://github.com/google-deepmind/mujoco/issues/1612

They both in some ways answers my question, I'll consider closing this issue I opened soon as a result. Keeping it open for now until I finish this task. For reference this approach suggested in the closed issues is what I am going to try:

The easiest way (but also hackiest way) to pass mjModel back to user-authored C++ code is to pass m._address to a function that accepts a uintptr_t as argument, reinterpret_cast it back to mjModel*, then pass it along to your actual function.

[peterdavidfagan]

Closing for now as I went with the approach outlined in existing issues. Initial code is here but its kind of messy.