How to use noise model of pyrobot in habitat?

[GuoPingPan]

Hi，I want to know how to use the pyrobot_noisy_controls.py in habitat-sim，it need pyrobot，then I just install it by

pip install pyrobot

and run the code comes the error as below

ModuleNotFoundError: No module named 'pyrobot.forms'

then I think may be I have not install entire pyrobot package, so I decide to down it following this repository, but I actually don't need to run in ros, so I only want to build the python part of the package, so I run below code

# my python env is python3
git clone --recurse-submodules https://github.com/facebookresearch/pyrobot.git
cd pyrobot/
pip install .

the process had done successfully! but when I try to import pyrobot, come the error below, I know it is because the tf in ros and conflict with python3

>>> import pyrobot.forms
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pgp/anaconda3/envs/eai/lib/python3.8/site-packages/pyrobot/__init__.py", line 6, in <module>
    from pyrobot.core import Robot
  File "/home/pgp/anaconda3/envs/eai/lib/python3.8/site-packages/pyrobot/core.py", line 18, in <module>
    import tf
  File "/opt/ros/melodic/lib/python2.7/dist-packages/tf/__init__.py", line 30, in <module>
    from tf2_ros import TransformException as Exception, ConnectivityException, LookupException, ExtrapolationException
  File "/opt/ros/melodic/lib/python2.7/dist-packages/tf2_ros/__init__.py", line 38, in <module>
    from tf2_py import *
  File "/opt/ros/melodic/lib/python2.7/dist-packages/tf2_py/__init__.py", line 38, in <module>
    from ._tf2 import *
ImportError: dynamic module does not define module export function (PyInit__tf2)

pyrobot #94 provide the method is that source the workspace, but I don't want to use ros, if there is any other easy way to install pyrobot for habitat?

[GuoPingPan]

And I found that there is a difference between the version of habitat-sim in the main branch(0.2.3) and the version in habitat-challenge(0.1.x)，but it can use noisy model in habitat-challenge and how to use it in version-0.2.3?

There is also a big different between the config system between the version 0.2.3 and 0.1.x.

[GuoPingPan]

I find the answer in https://github.com/facebookresearch/habitat-sim/blob/main/tests/test_pyrobot_noisy_controls.py but in habitat-lab I have make self-defined action pyrobot_noisy_actions.py just like:

import habitat
import habitat_sim
import habitat_sim.utils
from habitat.sims.habitat_simulator.actions import (
    HabitatSimV0ActionSpaceConfiguration, # else includes V1、PyRobot、VelocityCtrl
    HabitatSimActions # is an instantiation of the HabitatSimActionsSingleton singleton
)
from habitat.core.embodied_task import SimulatorTaskAction



@habitat.registry.register_action_space_configuration(name='MyPyrobot')
class CustomActionSpaceConfiguration(HabitatSimV0ActionSpaceConfiguration):
    def __init__(self, config):
        super().__init__(config)
        if not HabitatSimActions.has_action("NOISY_FORWARD"):
            HabitatSimActions.extend_action_space("NOISY_FORWARD")  # 16
        if not HabitatSimActions.has_action("NOISY_RIGHT"):
            HabitatSimActions.extend_action_space("NOISY_RIGHT")  # 17
        if not HabitatSimActions.has_action("NOISY_LEFT"):
            HabitatSimActions.extend_action_space("NOISY_LEFT")  # 18

    def get(self):
        config = super().get()

        config[HabitatSimActions.NOISY_FORWARD] = habitat_sim.ActionSpec(
            "pyrobot_noisy_move_forward",
            habitat_sim.PyRobotNoisyActuationSpec(
                amount=self.config.forward_step_size,
                robot='LoCoBot',
                controller='Proportional',
                noise_multiplier=0.5
            ),
        )
        config[HabitatSimActions.NOISY_RIGHT] = habitat_sim.ActionSpec(
            "pyrobot_noisy_turn_right",
            habitat_sim.PyRobotNoisyActuationSpec(
                amount=self.config.turn_angle,
                robot='LoCoBot',
                controller='Proportional',
                noise_multiplier=0.5
            ),
        )
        config[HabitatSimActions.NOISY_LEFT] = habitat_sim.ActionSpec(
            "pyrobot_noisy_turn_left",
            habitat_sim.PyRobotNoisyActuationSpec(
                amount=self.config.turn_angle,
                robot='LoCoBot',
                controller='Proportional',
                noise_multiplier=0.5
            ),
        )

        return config

@habitat.registry.register_task_action
class PyrobotNoisyMoveForwardAction(SimulatorTaskAction):
    def _get_uuid(self, *args, **kwargs):
        return "noisy_forward"

    def step(self, *args, **kwargs):

        return self._sim.step(HabitatSimActions.NOISY_FORWARD)

@habitat.registry.register_task_action
class PyrobotNoisyTurnRightAction(SimulatorTaskAction):
    def _get_uuid(self, *args, **kwargs):
        return "noisy_right"

    def step(self, *args, **kwargs):

        return self._sim.step(HabitatSimActions.NOISY_RIGHT)

@habitat.registry.register_task_action
class PyrobotNoisyTurnLeftAction(SimulatorTaskAction):
    def _get_uuid(self, *args, **kwargs):
        return "noisy_left"

    def step(self, *args, **kwargs):

        return self._sim.step(HabitatSimActions.NOISY_LEFT)

and before constructing env, you should modify the config and add the noisy action,

1. first import the module

import pyrobot_noisy_actions

2. add the actions into config

with habitat.config.read_write(config):
    config.habitat.simulator.action_space_config = 'MyPyrobot'
    config.habitat.task.actions['noisy_forward'] = ActionConfig(type='PyrobotNoisyMoveForwardAction')
    config.habitat.task.actions['noisy_right'] = ActionConfig(type='PyrobotNoisyTurnRightAction')
    config.habitat.task.actions['noisy_left'] = ActionConfig(type='PyrobotNoisyTurnLeftAction')

3. use the actions

env.step('noisy_forward')
env.step('noisy_right')
env.step('noisy_left')

I am not sure if it is a good way but act successfully and more easier. What's more, I want to know why I should import pyrobot_noisy_actions but no need to call any function or it will get Error? Is the import pyrobot_noisy_actions is to register the ActionConfig?