[Bug]: SO101 unable to pick up elastic object

[johnsutor]

Bug Description

I am working on a project involving teleoperation of a SO101 robot. I am using the Mujoco file as-is from the original repository. I can pick up a rigid cube just fine. However, I am unable to pick up an elastic cube as the grippers pass right through the object. I have tried a few different fixes to no avail, including

• Making the cube denser
• Using a larger target decimation face number
• Using all other solvers besides MPM
• Using MPM with ElastoPlastic material

Any guidance or help would be greatly appreciated! Please view the video below for more context.

Steps to Reproduce

import numpy as np
import genesis as gs
from lerobot.teleoperators.so101_leader import SO101LeaderConfig, SO101Leader

gs.init(backend=gs.gpu)

scene = gs.Scene(
    sim_options=gs.options.SimOptions(
        dt=1e-2,
        substeps=33,
    ),
    mpm_options=gs.options.MPMOptions(
        lower_bound=(-1.5, -1.5, -0.5),
        upper_bound=(1.5, 1.5, 1.5),
    ),
    rigid_options=gs.options.RigidOptions(
        box_box_detection=True,
    ),
    vis_options=gs.options.VisOptions(
        visualize_mpm_boundary=True,
    ),
    viewer_options=gs.options.ViewerOptions(
        camera_pos=(1, 1, 1),
        camera_lookat=(0.0, 0.0, 0.0),
        camera_fov=30,
        max_FPS=30,
    ),
    show_viewer=True,
)

cam = scene.add_camera(
    res=(1280, 960), pos=(1, 1, 1), lookat=(0.0, 0.0, 0.0), fov=30, GUI=False
)


teleop_config = SO101LeaderConfig(
    port="/dev/ttyACM0",
    id="sutie_leader",
    use_degrees=True,
)
teleop_device = SO101Leader(teleop_config)

plane = scene.add_entity(
    gs.morphs.Plane(),
)

so101 = scene.add_entity(
    gs.morphs.MJCF(
        file="assets/so101_new_calib.xml",
        pos=(0.0, 0.0, 0.0),
        euler=(0, 0, 0),
    ),
)

sponge = scene.add_entity(
    material=gs.materials.MPM.Elastic(),
    morph=gs.morphs.Box(
        size=(0.04, 0.04, 0.04),
        pos=(0.15, 0.15, 0.1),
        euler=(0, 0, 0),
    ),
    surface=gs.surfaces.Default(
        color=(1.0, 0.4, 0.4),
        vis_mode="visual",
    ),
)

joints = [
    "shoulder_pan",
    "shoulder_lift",
    "elbow_flex",
    "wrist_flex",
    "wrist_roll",
    "gripper",
]

dof_indices = [so101.get_joint(name).dof_idx for name in joints]

scene.build()
teleop_device.connect()


def convert_action_to_genesis(action):
    return np.array(
        [
            np.deg2rad(action["shoulder_pan.pos"]),
            np.deg2rad(action["shoulder_lift.pos"]),
            np.deg2rad(action["elbow_flex.pos"]),
            np.deg2rad(action["wrist_flex.pos"]),
            np.deg2rad(action["wrist_roll.pos"]),
            np.deg2rad(action["gripper.pos"]),
        ]
    )


cam.start_recording()

try:
    while True:
        action = teleop_device.get_action()
        dof_positions = convert_action_to_genesis(action)
        so101.control_dofs_position(dof_positions, dof_indices)
        cam.render()
        scene.step()
except Exception as e:
    print(f"Error: {e}")
finally:
    cam.stop_recording("./so101_teleop.mp4")

Expected Behavior

It should be able to grip and pick up the cube, with the cube deforming.

Screenshots/Videos

https://github.com/user-attachments/assets/c7f02a4c-1726-43bb-82f7-637b39ced1dc

Relevant log output

Environment

OS: Ubuntu 25.04 GPU: RTX 5090 CUDA Version: 12.8, CUDA Driver Version: 570.172.08 CPU: AMD 9950X Package Manager: uv Installed Packages: "genesis-world>=0.3.4", "lerobot[feetech]>=0.3.2", "ruff>=0.14.0", "torch>=2.8.0",

Release version or Commit ID

genesis-world>=0.3.4

Additional Context

No response

[johnsutor]

Hey! Just wanted to check in on this issue, as I am still running into this problem with elastic objects. Happy to help debug or test any patches/updates if that'd be useful. Thanks for all the great work on Genesis!