On simulating Behavior tasks

[vantony1]

hi, I am trying to use OmniGibson to simulate Behavior tasks and seeing if I can use the action primitives to complete some of them. It seems like to use behavior tasks we must hand-code the list of objects that need to be rendered for each task. Is there any way to get the simulation to automatically load the objects that a given behavior task requires and then use action primitives to finish it?

The relevant example: action_primitives.wip_solve_behavior_task fails to run correctly at the moment.

/omnigibson-src/omnigibson/controllers/controller_base.py:134: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison if command_output_limits == "default" 2024-05-07 17:36:59 [14,630ms] [Warning] [omni.syntheticdata.plugin] SdRenderVarPtr missing valid input renderVar DistanceToCameraSDhost Traceback (most recent call last): File "/micromamba/envs/omnigibson/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/micromamba/envs/omnigibson/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/omnigibson-src/omnigibson/examples/action_primitives/wip_solve_behavior_task.py", line 64, in <module> main() File "/omnigibson-src/omnigibson/examples/action_primitives/wip_solve_behavior_task.py", line 42, in main env = og.Environment(configs=config) File "/omnigibson-src/omnigibson/utils/python_utils.py", line 90, in wrapper func(*values.args, **values.kwargs) File "/omnigibson-src/omnigibson/envs/env_base.py", line 77, in __init__ self.load() File "/omnigibson-src/omnigibson/envs/env_base.py", line 371, in load self._load_robots() File "/omnigibson-src/omnigibson/envs/env_base.py", line 242, in _load_robots self.scene.reset() File "/omnigibson-src/omnigibson/scenes/scene_base.py", line 493, in reset self.load_state(self._initial_state) File "/omnigibson-src/omnigibson/utils/python_utils.py", line 508, in load_state self._load_state(state=state) File "/omnigibson-src/omnigibson/scenes/scene_base.py", line 681, in _load_state self._registry.load_state(state=state, serialized=False) File "/omnigibson-src/omnigibson/utils/python_utils.py", line 508, in load_state self._load_state(state=state) File "/omnigibson-src/omnigibson/utils/registry_utils.py", line 340, in _load_state obj.load_state(state[obj.name], serialized=False) File "/omnigibson-src/omnigibson/utils/python_utils.py", line 508, in load_state self._load_state(state=state) File "/omnigibson-src/omnigibson/utils/registry_utils.py", line 340, in _load_state obj.load_state(state[obj.name], serialized=False) File "/omnigibson-src/omnigibson/utils/python_utils.py", line 508, in load_state self._load_state(state=state) File "/omnigibson-src/omnigibson/robots/manipulation_robot.py", line 1424, in _load_state for arm in state["ag_obj_constraint_params"].keys(): KeyError: 'ag_obj_constraint_params'