[Design Change] Enable Bringing Your Own Control Scheme

[MichaelrMentele]

A key limitation of https://github.com/huggingface/lerobot/blob/1a343c359101c7fe0099dc64c5fe867cfc4be898/lerobot/common/robot_devices/robots/manipulator.py#L211 is that it is tightly coupled to taking in a leader robot when all it really needs to take in is a publisher that provides target joint angles.

The downside of tight coupling to a leader robot is that it precludes infinite other control schemes that could provide those target joint angles, also the articulation and robot are a tightly coupled pair, we may as well package them as such for the robot.

Proposal

Instead of taking in a "Leader" take in a ArticulationProvider interface. This change should be invisible to the codebase since this is done by hydra .yaml anyway

Old ManipulatorRobot interface:

    robot = ManipulatorRobot(
        robot_type="koch",
        calibration_dir=".cache/calibration/koch",
        leader_arms=leader_arms,  # But I want to use a ps4 controller or ultraleap or XYZ
        follower_arms=follower_arms,
    )

We could just have the arm be packaged with it's articulation provider from the get go.

New interface:

    robot = ManipulatorRobot(
        robot_type="koch",
        calibration_dir=".cache/calibration/koch",
        arms=arms,
    )

The koch.yaml would change to:

arms:
  arm1:
    _target_: lerobot.common.robot_devices.motors.dynamixel.DynamixelMotorsBus
    port: /dev/tty.usbmodem575E0031751
    articulationProvider: <path to class or separate config>
    motors:
      # name: (index, model)
      shoulder_pan: [1, "xl330-m077"]
      shoulder_lift: [2, "xl330-m077"]
      elbow_flex: [3, "xl330-m077"]
      wrist_flex: [4, "xl330-m077"]
      wrist_roll: [5, "xl330-m077"]
      gripper: [6, "xl330-m077"]

For the first iteration we'd simply have the provider passed in be a DynamixelMotorsBus wrapped with a few read commands. Then we could start adding XYZ new providers from the community.

Seems like a relatively straightforward refactor, mild indirection added, for infinite extensibility for articulation control schemes.

[MichaelrMentele]

We still need to define the interface for the provider a bit, agree on names, but let me know if we agree on the direction first.

[Cadene]

Thanks for your suggestions! As you know, our project regarding real robot interfaces needs iterations. We prefer to wait a bit before refactoring. Instead, we want to focus on supporting real well a minimal set of use cases. We also want to make our code simple enough so that it is easy for people to import lerobot and add their own robots or interfaces on top! Let's see how it goes!

[thefaftek-git]

@Cadene I'm about 80% of the way through adding some basic gamepad controls for the SO-101. If I just provide an integration example on this, is it something that would be worth a PR to publish as an example?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity (6 months). It will be closed if no further activity occurs. Any change, comment or update to this issue will reset this count. Thank you for your contributions.