Design docs for manipulation

[j-rivero]

Fixes o3de/o3de-extras#151

Draft documents about the implementation of the manipulation feature for the o3de-ros2-gem. Mostly inspired by o3de/o3de-ros2-gem#162.

List of files:

• design.md: main intro file
• analysis.md: detailed description about what we understand as manipulation, how does it applies to the o3de-ros2-gem
• architecture.md: (TODO) class diagram for the implementation
• features.md: list of features desired for the manipulation feature
• manipulation.svg: informal diagram of data flow and interactions between o3de and ROS 2 for the different aspects of the manipulation.

I left thearchitecture.md to be done since I have no previous knowledge of the gem/o3de internals. Any guidance or help is very welcome. I left the PR as Draft by now but I can happy answer to any feedback on the current content.

[adamdbrw]

@j-rivero I remember we spoke about the architecture. Are you working on it or should I go ahead? I would start with implementing a skeleton of Components and classes to populate (we can divide smaller tasks easier then)

[adamdbrw]

@j-rivero Is this PR going to be ready for review?

[j-rivero]

@j-rivero Is this PR going to be ready for review?

I think that is mostly ready with the exception of the architecture. I can spend some time trying to create but we can probably go faster if someone with more experience in O3DE and the Gem code base create the initial draft.

[j-rivero]

For quick prototyping I have lightened the manipulation.svg diagram in order to leave only the essential modules. Moreover I specified the type of controller which could be used: indeed a velocity controller allows to control the kinematics of the robotic arm and the dynamics to be forgotten, so that to highlight the early implementation stage. Any feedback on this solution are appreciated.

Thanks @danielemorra98. I like the simplification and as an easy use case to implement the system and check if the design is working in the very basic looks like a great approach to me. Let me comment it internally with the team and back here if someone sees something to be fixed or to complete it.

How do you prefer to integrate it in this document? Adding to the current diagram as a variant of it or do you prefer to replace the original in this PR?

[lgleim]

How do you prefer to integrate it in this document? Adding to the current diagram as a variant of it or do you prefer to replace the original in this PR?

@j-rivero We were discussing that it could make sense to create an implementation roadmap section as part of the proposal. As @danielemorra98 suggested, this reduced design could be a good first implementation milestone. After a review, the next implementation milestone would then be defined, e.g. based on your original architecture proposal. Looking forward to your / your team's feedback!

[j-rivero]

I like the simplification and as an easy use case to implement the system and check if the design is working in the very basic looks like a great approach to me. Let me comment it internally with the team and back here if someone sees something to be fixed or to complete it.

Team agree that the simplification looks valid to our eyes. Good work.

While the existing popular toolset for manipulation around MoveIt is a great starting point for the open source community and outreach, e.g. to academic partners, the requirements of many industrial users may not be met.

While I don't disagree that in some cases many industrial use cases requirements are hard to implement with the current infrastructure we have people at Open Robotics successfully using it for industrial projects. Did you have any bad experience with respect to the use of it in your use cases?

we were discussing that it could make sense to create an implementation roadmap section as part of the proposal.

Sounds good to me. Could you please send a PR against this one or a patch that I can apply? Thanks so much.

[j-rivero]

As a request from @michalpelka, I've tried to document a whole integration between Gazebo + ros2_control + MoveIt. With this in hand, and regarding the controller side of things: maybe the long-term option is try to implement a GEM that replaces gazebo_ros2_control. This pkg implements a generic hardware_interface that is used by ros2 controllers like joint_trajectory_controller to know a) what the current joint values are and b) command joints to certain values. The ros2_control architecture which separates the controller and the hardware_interface allows for this nicely.

[adamdbrw]

@j-rivero @lgleim anything I can assist with to move this PR forward?

[j-rivero]

@j-rivero @lgleim anything I can assist with to move this PR forward?

Would be great to have the suggestions done by @lgleim and @danielemorra98 to the document, I can take care if they don't have time but would probably be more accurate if a PR is sent against this.

As a request from @michalpelka, I've tried to document a whole integration between Gazebo + ros2_control + MoveIt. With this in hand, and regarding the controller side of things: maybe the long-term option is try to implement a GEM that replaces gz_ros2_control. This pkg implements a generic hardware_interface that is used by ros2 controllers like joint_trajectory_controller to know a) what the current joint values are and b) command joints to certain values.

With the Gazebo example in hand, we probably want to add the integration with ros2_control to the document in some degree. I think that @michalpelka was interested into providing feedback for this.

[j-rivero]

Would be great to have the suggestions done by @lgleim and @danielemorra98 to the document, I can take care if they don't have time but would probably be more accurate if a PR is sent against this.

We submitted a PR for incorporating the feedback in https://github.com/RobotecAI/o3de-ros2-gem/pull/358. I'll wait until Dec 14th for merging it, @lgleim and @danielemorra98 just in case that you have time to give it a look.

[j-rivero]

We submitted a PR for incorporating the feedback in o3de/o3de-ros2-gem#358. I'll wait until Dec 14th for merging it,

Already merged.

[j-rivero]

We submitted a PR for incorporating the feedback in https://github.com/RobotecAI/o3de-ros2-gem/pull/358. I'll wait until Dec 14th for merging it, @lgleim and @danielemorra98 just in case that you have time to give it a look.

A bit late but don't want to leave this unfinished. @adamdbrw could you please merge it?

[danielemorra98]

I have just a couple of question about the current status.

• Looking at Switch vehicle dynamics to PhysX joint and Adding actuation to PhysX joints, it seems that they are switching to a dynamics model which suits better robotics applications. This demo uses the standard method to move joints (apply an angular/linear impulse to the rigid body). Are you planning to switch the current MotorizedJoint to an actuated joint?
• Are you working on a solution similar to the one used for the krakenDemo that also works with revolute joints? I think that it would be great to have another general controller for manipulation purposes to be chosen from the Ros2RobotControl component (as it happens with Twist or Ackermann labels)

Maybe this is not the right place to comment on this and if that I apologize.

[michalpelka]

Thanks, @danielemorra98 for the questions. Let me update you. The latest changes to the O3DE PhysX gem developed were intended for manipulation. This method of actuation is stable and easy to use, Motorized joint was a temporary solution that used available API. We are switching the existing codebase to use PhysX actuation. I also tested PhysX actuation against the manipulation task and performs well: https://github.com/RobotecAI/o3de-physics-test-scene. It is stable in application with PID controller and position control. Actuation PhysX allows us at this moment :

• apply velocity speed
• limit force / torque
• it works with Hinge and Prismatic joints.

[danielemorra98]

I would like to develop the bridge to connect the Gem to MoveIt with the latest PhysX API update (new Joint method) and the robotic arm found in o3de-physics-test-scene project. I found out that the requirements for a MoveIt interconnection are the following:

• /joint_states topic: notifies MoveIt with the current joints status
• tf: MoveIt monitors transform information (using ROS tf library) for internal use
• Controller interface: MoveIt talks to controllers using FollowJointTrajectoryAction ROS interface. Need to implement a ROS service to server this action.
• Sensor information: MoveIt represents the world building an occupancy map from PointClouds or DepthImages

Remarks:

• The first two points should be easy to implement or already there (tf).
• The last point is not necessary for first tests
• The core part is the Controller interface: develop a server to take into account a FollowJointTrajectoryAction is not straightforward (take inspiration from the ros2_controller implementation)
  • The server will have to implement whatever implementation scheme is needed for the robot to move "smoothly" (or whatever criteria you want to use). For example that could mean interpolation between the points making up the trajectory.

What are your comments and/or suggestion on this?

[j-rivero]

What are your comments and/or suggestion on this?

Thanks for the comments Daniele. I read briefly your comments and did not find any big conceptual error. I'm afraid that I've been moved to different projects and don't have time to follow up on this one, sorry. I'm pretty sure that @michalpelka or @adamdbrw can help you more on this.

[adamdbrw]

Could we move this to o3de-extras repos and merge in there?