mujoco
mujoco copied to clipboard
Published
20 hours ago •
google-deepmind
google-deepmind
Grasping cloth object using Franka gripper
Hello,
I have flex object with a definition like
Details
<?xml version="1.0" encoding="utf-8"?>
<mujoco model="cloth">
<compiler angle="radian"></compiler>
<option timestep="0.01" tolerance="1e-06" viscosity="0.3" integrator="implicitfast" solver="CG"></option>
<size nkey="10"></size>
<statistic meansize="0.0703541" extent="2" center="0 0 0"></statistic>
<extension>
<plugin plugin="mujoco.elasticity.shell">
<instance name="cloth_flexcomp_cloth">
<config key="face" value="0 5 6 0 6 1 1 6 7 1 7 2 2 7 8 2 8 3 3 8 9 3 9 4 5 10 11 5 11 6 6 11 12 6 12 7 7 12 13 7 13 8 8 13 14 8 14 9 10 15 16 10 16 11 11 16 17 11 17 12 12 17 18 12 18 13 13 18 19 13 19 14 15 20 21 15 21 16 16 21 22 16 22 17 17 22 23 17 23 18 18 23 24 18 24 19"></config>
<config key="edge" value="0 1 2 3 4 1 5 6 3 7 8 6 9 10 7 11 12 10 13 14 11 15 16 14 17 18 19 20 0 18 21 22 20 23 5 22 24 25 23 26 9 25 27 28 26 29 13 28 30 31 32 33 17 31 34 35 33 36 21 35 37 38 36 39 24 38 40 41 39 42 27 41 43 44 45 46 30 44 47 48 46 49 34 48 50 51 49 52 37 51 53 54 52 55 40 54 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"></config>
<config key="young" value="85242.02369946481"></config>
<config key="poisson" value="0.4032740512363587"></config>
<config key="thickness" value="0.0005"></config>
</instance>
</plugin>
</extension>
<worldbody>
<geom name="floor" size="0 0 0.1" type="plane"></geom>
<light pos="0 0 4" dir="0 0 -1" diffuse="0.6 0.6 0.6" specular="0.2 0.2 0.2"></light>
<body>
<body name="object" pos="0 0 0.83">
<site name="bottom_site" pos="0 0 -0.045" rgba="0 0 0 0"></site>
<site name="top_site" pos="0 0 0.03" rgba="0 0 0 0"></site>
<site name="horizontal_radius_site" pos="0.03 0.03 0" rgba="0 0 0 0"></site>
<body name="cloth_0" pos="0.17 -0.038 0.00076">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_1" pos="0.13 -0.057 0.00067">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_2" pos="0.078 -0.059 0.00076">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_3" pos="0.072 -0.0088 0.00066">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_4" pos="0.022 -0.017 0.00065">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_5" pos="0.13 -0.06 0.0044">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_6" pos="0.11 -0.02 0.03">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_7" pos="0.074 -0.01 0.00085">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_8" pos="0.023 -0.016 0.0026">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_9" pos="0.013 0.032 0.00075">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_10" pos="0.11 -0.085 0.046">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_11" pos="0.077 -0.05 0.05">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_12" pos="0.043 -0.012 0.04">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_13" pos="0.019 0.029 0.023">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_14" pos="0.061 0.045 0.00067">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_15" pos="0.086 -0.098 0.005">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_16" pos="0.076 -0.049 0.00047">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_17" pos="0.07 -0.00053 0.00052">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_18" pos="0.061 0.046 0.0027">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_19" pos="0.045 0.093 0.00091">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_20" pos="0.14 -0.089 0.00076">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_21" pos="0.13 -0.04 0.00075">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_22" pos="0.12 0.0095 0.00076">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_23" pos="0.11 0.059 0.00076">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
<body name="cloth_24" pos="0.094 0.11 0.00075">
<inertial pos="0 0 0" mass="0.02" diaginertia="3.33333e-07 3.33333e-07 3.33333e-07"></inertial>
<joint pos="0 0 0" axis="1 0 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 1 0" type="slide"></joint>
<joint pos="0 0 0" axis="0 0 1" type="slide"></joint>
<plugin instance="cloth_flexcomp_cloth"></plugin>
</body>
</body>
</body>
</worldbody>
<deformable>
<flex name="cloth" radius="0.001" body="cloth_0 cloth_1 cloth_2 cloth_3 cloth_4 cloth_5 cloth_6 cloth_7 cloth_8 cloth_9 cloth_10 cloth_11 cloth_12 cloth_13 cloth_14 cloth_15 cloth_16 cloth_17 cloth_18 cloth_19 cloth_20 cloth_21 cloth_22 cloth_23 cloth_24" element="0 5 6 0 6 1 1 6 7 1 7 2 2 7 8 2 8 3 3 8 9 3 9 4 5 10 11 5 11 6 6 11 12 6 12 7 7 12 13 7 13 8 8 13 14 8 14 9 10 15 16 10 16 11 11 16 17 11 17 12 12 17 18 12 18 13 13 18 19 13 19 14 15 20 21 15 21 16 16 21 22 16 22 17 17 22 23 17 23 18 18 23 24 18 24 19">
<contact selfcollide="auto" condim="4" friction = "4 0.1 0.001" solimp="0.001"></contact>
<!-- <edge damping="1"></edge>-->
</flex>
</deformable>
<equality>
<!-- <flex flex="cloth"></flex>-->
<flex flex="cloth_cloth"></flex>
</equality>
</mujoco>
I am using robosuite which has a model for Franka Gripper with the definition like
Details
<body name="leftfinger" pos="0 0 0.0524" quat="0.707107 0 0 0.707107">
<inertial pos="0 0 0.05" mass="0.1" diaginertia="0.01 0.01 0.005" />
<joint name="finger_joint1" pos="0 0 0" axis="0 1 0" type="slide" limited="true" range="0.0 0.04" damping="100" armature="1.0" frictionloss="1.0"/>
<geom type="mesh" contype="0" conaffinity="0" group="1" mesh="finger_vis" name="finger1_visual" rgba="0.499 0.499 0.499 1" />
<geom type="mesh" group="0" conaffinity="1" contype="0" solref="0.02 1" friction="1 0.005 0.0001" condim="4" mesh="finger" name="finger1_collision"/>
<body name="finger_joint1_tip" pos="0 0.0085 0.056">
<inertial pos="0 0 0" quat="0 0 0 1" mass="0.01" diaginertia="0.01 0.01 0.01" />
<geom size="0.008 0.004 0.008" pos="0 -0.005 -0.015" quat="0 0 0 1" type="box" group="0" solref="0.01 0.5" friction = "2 0.05 0.0001" conaffinity="1" contype="1" name="finger1_pad_collision"/>
</body>
</body>
<body name="rightfinger" pos="0 0 0.0524" quat="0.707107 0 0 0.707107">
<inertial pos="0 0 0.05" mass="0.1" diaginertia="0.01 0.01 0.005" />
<joint name="finger_joint2" pos="0 0 0" axis="0 1 0" type="slide" limited="true" range="-0.04 0.0" damping="100" armature="1.0" frictionloss="1.0"/>
<geom quat="0 0 0 1" type="mesh" contype="0" conaffinity="0" group="1" mesh="finger_vis" name="finger2_visual" rgba="0.499 0.499 0.499 1" />
<geom quat="0 0 0 1" type="mesh" group="0" conaffinity="1" contype="0" solref="0.02 1" friction="1 0.005 0.0001" condim="4" mesh="finger" name="finger2_collision"/>
<body name="finger_joint2_tip" pos="0 -0.0085 0.056">
<inertial pos="0 0 0" quat="0 0 0 1" mass="0.01" diaginertia="0.01 0.01 0.01" />
<geom size="0.008 0.004 0.008" pos="0 0.005 -0.015" quat="0 0 0 1" type="box" group="0" solref="0.01 0.5" friction = "2 0.05 0.0001" conaffinity="1" contype="1" name="finger2_pad_collision"/>
</body>
</body>
I have tried loads of contact parameters but I cannot get the cloth to stick to the gripper when grasping. You can see the failed video here
Does anyone know what parameters could work? These fingers can reliably grasp a cube whose solimp=[0.99, 0.99, 0.01], and solref=[0.01, 1]. I am using a very simple cloth object with only 25 bodies(it was automatically generated by flexcomp macro) because my laptop is too slow to simulate more detailed cloth.
I have experienced the same when trying to grasp cloth-like objects with a robotic gripper. Rather than this being related to the cloth parameters I wonder if it has something to do with the model itself.
I guess @quagla can point to what can this issue relate to and whether this is something the team is working on addressing.
Hi, I cannot tell from the video what exactly the issue is. Do you still have the same problem if you make the cloth thicker by increasing radius in the flex parameters?
I tried by increasing the radius + different contact parameters and the issue remains:
- The franka applies a constant force trying to close the grippers for grasping the object, and when moving the franka to a new position with the cloth "grasped" the cloth slips from the fingers of the robot. In the same environment grasping works well with a rigid object (box).