[Question] No module named 'isaacsim.robot'

[redzhepdx]

IsaacLab can't access to isaacsim.robot during headless training

I am currently running an experiment using my custom robot and custom grippers. I have implemented the DirectRLEnv as demonstrated in the tutorials and I can start the policy training without --headless but when I use headless I am getting the following error:

from isaacsim.robot.surface_gripper._surface_gripper import Surface_Gripper, Surface_Gripper_Properties
ModuleNotFoundError: No module named 'isaacsim.robot'

I didn't change anything in the code and non-headless version works as expected but headless version crashes as I described. One thing I noticed, I also have other imports from isaac-sim like

from isaacsim.core.prims import XFormPrim
from omni.physics.tensors import SimulationView, Transform  # type: ignore # pylint: disable=no-name-in-module

and they are being imported as expected. I don't know if this is a bug or I am doing something wrong.

Using IsaacLab 2.0 and IsaacSim 4.5(local non pip installation).

[RandomOakForest]

Thanks for posting this. Are you doing the imports after the launching the app? That is, you need to have the following before your imports:

import argparse

from isaaclab.app import AppLauncher

# create argparser
parser = argparse.ArgumentParser(description="Tutorial on creating an empty stage.")
# append AppLauncher cli args
AppLauncher.add_app_launcher_args(parser)
# parse the arguments
args_cli = parser.parse_args()
# launch omniverse app
app_launcher = AppLauncher(args_cli)
simulation_app = app_launcher.app

"""Rest everything follows."""

from isaaclab.sim import SimulationCfg, SimulationContext

[redzhepdx]

As I said in the explanation, I can start the rl_games training without headless, this error happens when I run it headless. I checked the AppLauncher's initialization and import error multiple times and I am sure that surface_gripper is being initialized after it. I can't identify where this error is coming from since it works for other isaacsim imports.

[redzhepdx]

Hi @RandomOakForest,

I've been looking into some related issues like #1850 and #126 and saw that there's ongoing development mentioned for enabling surface grippers in a parallelized setup. This is a key feature for my daily work, and I imagine it's a common problem for many others relying on similar gripper types.

Currently, the grasping doesn't happen in GPU mode and it forces me to run all simulation on CPU, which introduces a major performance bottleneck. When combined with the surface gripper limitations, it significantly slows down development and experimentation.

Is parallelized surface gripper support something planned for an upcoming release? If not, are there any recommended workarounds or alternative approaches for simulating surface or suction grippers in the meantime?

Appreciate the support!

[RandomOakForest]

Thank you for following up on this. There is current work by the team on the surface gripper with GPU support. It is hard to say when this will be available, but it is in our current roadmap. I will move this issue to our Discussions for follow up.