isaac-sim
overlap_mesh ignores updated joint positions in IsaacLab
I'm using overlap_mesh with an articulation to detect collisions, however, it's behaving incorrectly.
When running the same setup in Isaac Sim’s Script Editor, the collision detection reflects the robot's current joint positions. However, in IsaacLab, using a simple implementation adapted from the rigid object tutorial, the function appears to ignore the updated joint configurations and always uses the default pose defined in the USD (all joints at 0).
For example, if a cube overlaps the arm only in the default configuration, a collision is reported even when the robot is spawned with the arm away. This suggests that IsaacLab does not update the articulation's state for overlap_mesh. Is there a step I’m missing to force the collision check to use the current joint positions, or is this a known limitation?
Default configuration:
Arm in a different configuration:
Output (Equal to both):
Doing the opposite—placing the cube so it overlaps only in the updated configuration—produces no collision detection in either case.
Config:
TIAGO_COLLISION_CFG = ArticulationCfg(
spawn=sim_utils.UsdFileCfg(
usd_path=USD_PATH,
),
init_state=ArticulationCfg.InitialStateCfg(
pos=(0.0, 0.0, 0.0),
joint_pos={
"arm_1_joint": 0.5689773361501514,
"arm_2_joint": -0.9791297103688189,
"arm_3_joint": -1.6266468628587152,
"arm_4_joint": 1.9879300180215413,
"arm_5_joint": -0.0715584993317675,
"arm_6_joint": 0.19722220547535926,
"arm_7_joint": -0.04363323129985824,
},
),
actuators={...}
)
code:
counter = 0
def report_hit(collision):
global counter
counter+=1
print("Collision with: ", collision.rigid_body)
return True #False stops search
def check_overlapping(path):
global counter
counter = 0
path_tuple = PhysicsSchemaTools.encodeSdfPath(path)
_ = get_physx_scene_query_interface().overlap_mesh(path_tuple[0], path_tuple[1],report_hit, False)
if counter > 0:
print(path)
def design_scene():
"""Designs the scene."""
cfg = sim_utils.GroundPlaneCfg()
cfg.func("/World/defaultGroundPlane", cfg)
stage = omni.usd.get_context().get_stage()
omni.kit.commands.execute('AddPhysicsSceneCommand',stage = stage, path='/World/PhysicsScene')
prim_utils.create_prim(f"/World/Origin0", "Xform")
cube_cfg = sim_utils.CuboidCfg(
size=(0.3043, 0.3941, 0.18363),
collision_props=sim_utils.CollisionPropertiesCfg(),
visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(0.0, 1.0, 0.0), metallic=0.2),
)
tiago_collision_cfg = TIAGO_COLLISION_CFG.replace(prim_path="/World/Origin0/Tiago")
tiago = Articulation(cfg=tiago_collision_cfg)
cube_object = cube_cfg.func("/World/Origin0/cube", cube_cfg, translation=(0.1,-0.5, 0.8))
scene_entities = {"tiago": tiago}
return scene_entities
def run_simulator(sim: sim_utils.SimulationContext, entities):
"""Runs the simulation loop."""
tiago = entities["tiago"]
sim_dt = sim.get_physics_dt()
sim_time = 0.0
count = 0
stage = omni.usd.get_context().get_stage()
paths = [res.string for prim in stage_utils.traverse_stage() if (res:=re.search(".*collisions/mesh.*", prim_utils.get_prim_path(prim)))]
while simulation_app.is_running():
if count % 250 == 0:
sim_time = 0.0
count = 0
#Position
root_state_tiago = tiago.data.default_root_state.clone()
tiago.write_root_pose_to_sim(root_state_tiago[:, :7])
tiago.write_root_velocity_to_sim(root_state_tiago[:, 7:])
#joints
joint_pos, joint_vel = tiago.data.default_joint_pos.clone(), tiago.data.default_joint_vel.clone()
tiago.write_joint_state_to_sim(joint_pos, joint_vel)
tiago.set_joint_position_target(joint_pos)
tiago.reset()
print("----------------------------------------")
print("[INFO]: Resetting object state...")
tiago.write_data_to_sim()
sim.step()
if count % 100 == 0:
for path in paths:
check_overlapping(path)
sim_time += sim_dt
count += 1
# update buffers
tiago.update(sim_dt)
Thank you for posting this. Could you clarify if the images you posted are before stepping the simulation or after how many steps? Also, please share the versions of Isaac Sim and Isaac Lab you are using. Thanks.
Hi, Thanks for the quick reply. I'm using Isaac Sim 4.5.0 and Isaac Lab 2.0.2. Both images were taken after stepping the simulation. In my setup, collisions are checked every 100 simulation steps using the code below:
sim.step()
if count % 100 == 0:
for path in paths:
check_overlapping(path)
sim_time += sim_dt
count += 1
To clarify, the images I posted are from two separate runs of the code shown in the initial post, not from the same simulation instance. The only difference between the two runs is the joint_pos field defined in the robot's config (ArticulationCfg.InitialStateCfg).
In one run, all joint positions were set to 0, which corresponds to the image where the arm is extended. This is also the default configuration when I spawn the USD directly in Isaac Sim.
In the other run, the robot config had the following joint positions:
joint_pos={
"arm_1_joint": 0.5689773361501514,
"arm_2_joint": -0.9791297103688189,
"arm_3_joint": -1.6266468628587152,
"arm_4_joint": 1.9879300180215413,
"arm_5_joint": -0.0715584993317675,
"arm_6_joint": 0.19722220547535926,
"arm_7_joint": -0.04363323129985824,
}
I didn’t move the arm dynamically during the simulation. Instead, I set the pose during reset using:
root_state_tiago = tiago.data.default_root_state.clone()
tiago.write_root_pose_to_sim(root_state_tiago[:, :7])
tiago.write_root_velocity_to_sim(root_state_tiago[:, 7:])
#joints
joint_pos, joint_vel = tiago.data.default_joint_pos.clone(), tiago.data.default_joint_vel.clone()
tiago.write_joint_state_to_sim(joint_pos, joint_vel)
tiago.set_joint_position_target(joint_pos)
All of the above code snippets are included in the code of the original post, should you need further context.
Following up on this, it seems some shapes cannot be checked with overlap_mesh. I'm tagging this as a bug report for the team to follow up. A workaround is to use a shape that has mesh you can visualize.
A workaround is to use a shape that has mesh you can visualize.
What do you mean by a mesh I can visualize? Both the cube and the arm meshes are visible.