Habitat 2.0 ArmEEAction Does not run

[mihdalal]

Habitat-Lab and Habitat-Sim versions

Habitat-Lab: v0.2.1

Habitat-Sim: v0.2.1

Habitat is under active development, and we advise users to restrict themselves to stable releases of Habitat-Lab and Habitat-Sim. The bug you are about to report may already be fixed in the latest version.

Master branch contains 'bleeding edge' code, but we do appreciate bug reports for it!

🐛 Bug

I tried to switch the controller for the mobile manipulator from joint position control to end-effector and I find that the ik object itself is not found in the simulator.

Steps to Reproduce

Steps to reproduce the behavior: Change Line 96 in rearrangepick_replica_cad.yaml to
ARM_CONTROLLER: "ArmEEAction" Then run python examples/example.py

Output:

Traceback (most recent call last):
  File "examples/example.py", line 30, in <module>
    example()
  File "examples/example.py", line 19, in example
    observations = env.reset()  # noqa: F841
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/core/env.py", line 222, in reset
    observations = self.task.reset(episode=self.current_episode)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/rearrange_pick_task.py", line 140, in reset
    super().reset(episode)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/rearrange_task.py", line 54, in reset
    observations = super().reset(episode)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/core/embodied_task.py", line 288, in reset
    action_instance.reset(episode=episode, task=self)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/actions.py", line 65, in reset
    self.arm_ctrlr.reset(*args, **kwargs)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/actions.py", line 351, in reset
    arm_pos = self.set_desired_ee_pos(np.array([0.5, 0.0, 1.0]))
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/actions.py", line 377, in set_desired_ee_pos
    ik.set_arm_state(joint_pos, joint_vel)
AttributeError: 'NoneType' object has no attribute 'set_arm_state'

Expected behavior

Run 200 steps in the environment without erroring.

[dhruvbatra]

CC: @ASzot

[ASzot]

Do you have pybullet installed? I believe that's the issue here.

[mihdalal]

Yes, I have pybullet installed already. version 3.0.4 Any thoughts on what else the issue could be? Does it work fine on your end?

[guhur]

I think you just ot load the URDF as in examples/interactive_play.py:

config.SIMULATOR.IK_ARM_URDF = (
    "./data/robots/hab_fetch/robots/fetch_onlyarm.urdf"
)

[ASzot]

By the way, this issue should now be resolved on main.

Also, the hab_suite branch now contains most of the functionality from the paper and we are slowly merging this into main. We made this tutorial to help get started with Habitat 2.0.

Please let me know if you have any more questions!

[mihdalal]

I just tried the hab_suite branch and it seems to have the same issue, specifically self._sim.ik_helper is None Main has the same error as well:

Traceback (most recent call last):
  File "examples/example.py", line 28, in <module>
    example()
  File "examples/example.py", line 17, in example
    observations = env.reset()  # noqa: F841
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/core/env.py", line 255, in reset
    observations = self.task.reset(episode=self.current_episode)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/sub_tasks/pick_task.py", line 144, in reset
    super().reset(episode)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/rearrange_task.py", line 70, in reset
    observations = super().reset(episode)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/core/embodied_task.py", line 288, in reset
    action_instance.reset(episode=episode, task=self)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/actions.py", line 78, in reset
    self.arm_ctrlr.reset(*args, **kwargs)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/actions.py", line 333, in reset
    cur_ee = self._sim.ik_helper.calc_fk(
AttributeError: 'NoneType' object has no attribute 'calc_fk'
[19:34:16:986295]:[Sim] Simulator.cpp(65)::~Simulator : Deconstructing Simulator

[ASzot]

That is surprising. I just tried and it works for me. Here is the exact command I ran python examples/interactive_play.py --play-task --never-end --add-ik, does that exact command work for you? I think you are still somehow using an old interactive play script since it should also print an error when PyBullet is not configured. Below is are the outputs I see for a fresh install:

[mihdalal]

Okay using your command things seem to be working now: I was running python examples/example.py and that does not seem to work. I am assuming that things work in this new script due to --add-ik flag.

[rpartsey]

@mihdalal Feel free to re-open the issue, if you still have questions.

[vantony1]

I just tried the hab_suite branch and it seems to have the same issue, specifically self._sim.ik_helper is None Main has the same error as well:

Traceback (most recent call last):
  File "examples/example.py", line 28, in <module>
    example()
  File "examples/example.py", line 17, in example
    observations = env.reset()  # noqa: F841
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/core/env.py", line 255, in reset
    observations = self.task.reset(episode=self.current_episode)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/sub_tasks/pick_task.py", line 144, in reset
    super().reset(episode)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/rearrange_task.py", line 70, in reset
    observations = super().reset(episode)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/core/embodied_task.py", line 288, in reset
    action_instance.reset(episode=episode, task=self)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/actions.py", line 78, in reset
    self.arm_ctrlr.reset(*args, **kwargs)
  File "/home/mdalal/research/mobile_manip/habitat-lab/habitat/tasks/rearrange/actions.py", line 333, in reset
    cur_ee = self._sim.ik_helper.calc_fk(
AttributeError: 'NoneType' object has no attribute 'calc_fk'
[19:34:16:986295]:[Sim] Simulator.cpp(65)::~Simulator : Deconstructing Simulator

I am having the same issue now. while trying to build on articulated_agents_tutorial.ipynb in the latest version of the repo. @rpartsey @ASzot do you have any ideas on how to fix this? essentially, my env.sim.ik_helper is None always.