huggingface
ros2_control support
Hello,
I was thinking that it would be great to use the robot with ros2_control :
- to test code developped with the ROS2 framework:
- for education purposes : the robot is great, easily and not expensive to build (thank you for the work achieved), transporteable in a case, etc.
Do you have any knowledge of an existing project ? If not, would you be interested in this kind of implementation ?
Best, Aline
I've added LeRobot support for MoveIt2 which uses ros2_control underneath the hood: https://github.com/huggingface/lerobot/pull/1280, maybe that will be of interest?
@baaluidnrey having ros2_control integrated would be amazing!
@ycheng517 it's nice to see your PR! I'm wondering if it might be better to integrate by talking joint_state_broadcaster/JointStateBroadcaster, position_controllers/JointGroupPositionController, position_controllers/GripperActionController controllers directly as opposed to integrating via moveit2.
We recently merged https://github.com/JafarAbdi/ros2_so_arm100/pull/15 to bringup a leader & follower SO100 arm using ros2_controls and I can see your PR bridging the gap to lerobot easily if it could skip the moveit2 layer since it simply forwards commands to the same controllers. The bringup package linked above uses this feetech_ros2_driver.
I'd be happy to collaborate on an effort to get this functionality merged 😄
@Yadunund, I think a direct ros2_control integration would be valuable! I like MoveIt as it provides Cartesian space position and velocity control which is more friendly for controlling the robot with keyboard or gamepad, as well as collision checking, which allows you to specify surfaces that the robot should not crashing into when executing a policy. Maybe we can incorporate both approaches. Would be happy to collaborate further!
@ycheng517 there s also a ROS 2 gello package https://github.com/wuphilipp/gello_software/tree/main/ros2 and a spacemouse one https://github.com/olaghattas/franka_teleop/tree/master/franka_teleop , those are also straightforwards way to teleop.
I have been uisng a UR10e and the https://github.com/isri-aist/RoboManipBaselines suites to teleop/train (spacemouse) but it would be nice to use ros2. I should have build a gello for the UR next week, I could try to test/help as well if you guys have something in mind.
@Yadunund I've adapted my PR to support joint position control with ros2_control using the components you mentioned.
@yoann-fleytoux testing with the UR would be very welcome, would be interested to see how that fits with the interfaces I've defined and Lerobot! Gello and spacemouse sound like great hardware, I wish I had those.
@ycheng517 Just like @yoann-fleytoux described, I’ve written a simple ROS 2 wrapper, following (GELLO)'s implementation for franka robot to control UR robots via the UR ROS 2 Driver (which uses ros2_control). You can find my fork here: 👉 https://github.com/JemuelStanley47/gello_software_UR/tree/jemprem/ur
My next step is to add UR robot configurations within LeRobot (like: https://github.com/huggingface/lerobot/issues/575) Here's a short video showing control of a UR5e in Isaac Sim using a physical Gello setup:
🎥 URSim Demo
🎥 IsaacSim Demo
Your repository looks quite relevant to what I’m working on—I’ll explore your interfaces and follow up with any updates!
@JemuelStanley47 cool, let me know how you end up integrating ROS with Lerobot!
Since it looks like my PR won't be reviewed, I've created a simple repo for it, feel free to check it out: https://github.com/ycheng517/lerobot-ros
@ycheng517 We also have been trying to combine ROS2 with lerobot. Nice that you also integrated moveit with lerobot!
At our lab we have implemented robot agnostic torque-based cartesian controllers (osc and cartesian impedance) for ros2_control and we built a gym environment on top of it to train VLAs and learning-based policies in general. We have been using lerobot to train policies and we have documented it in our project website.
We decided to skip using MoveIt to make the interface a bit easier - just ros2_controllers that subscribe to target_pose / target_joints - and also to use the nice compliant properties of a cartesian impedance controller. The gym however, should also work with position/velocity controlled robots, given that they also provide a topic to publish target poses or joints. We use the interface with our Franka Robotics FR3 and it has been working quite good.
Love to hear some feedback if you try it out, cheers!
@danielsanjosepro the CRISP project looks really impressive!
Do you think it would be possible to combine a cartesian leader (e.g. a VR controller) with a joint space policy?
As far as I understand the documentation, the CRISP controllers do not publish the desired joint positions based on the EE target pose, as they compute $\boldsymbol{\tau}_{\text{task}}$ based on the error in cartesian space. However, most VLA policies require the current actions in joint space.
LeRobot currently avoids this problem by integrating a separate IK pipeline, which is not as responsive.
@Nico0302 hey! You are right, we have been mainly doing cartesian control and if needed a robot to robot teleop :) i think that the IK solution would be great. You could use the URDF pf the robot and then that would be fine! If you want to, feel free to colaborate to the repo!