Question about “scene_dataset_config” for "semantic_sensor"

[Cepillar]

Habitat version

habitat-lab-0.3.0 habitat-sim-0.3.0

❓ Questions and Help

I encountered the same issue as in #1808 when using mp3d dataset, and I have tried setting SIMULATOR.SCENE_DATASET to "data/scene_datasets/mp3d/mp3d.scene_dataset_config.json", but it didn't work. My vln_r2r.yaml file is as follows:

# @package _global_

defaults:
  - /habitat: habitat_config_base
  - /habitat/task: vln_r2r
  - /habitat/simulator/[email protected]_agent: 
    - rgbds_agent
  - /habitat/dataset/vln: mp3d_r2r
  - _self_

habitat:
  environment:
    max_episode_steps: 500
  simulator:
    agents:
      main_agent:
        sim_sensors:
          rgb_sensor:
            width: 1280
            height: 640
            # hfov: 90
            position: [0, 0.88, 0]
            # type: HabitatSimRGBSensor
          depth_sensor:
            width: 481
            height: 241
            min_depth: 0.0
            max_depth: 10.0
            position: [0, 0.88, 0]
            # type: HabitatSimDepthSensor
          semantic_sensor:
            width: 1280
            height: 640
            position: [0, 0.88, 0]
    forward_step_size: 0.25
    turn_angle: 15
    scene_dataset: "data/scene_datasets/mp3d/mp3d.scene_dataset_config.json"
    habitat_sim_v0:
      gpu_device_id: 0

I printed the config after calling habitat.get_config. In the config, scene_dataset has been correctly set to "data/scene_datasets/mp3d/mp3d.scene_dataset_config.json". However, when I print env.episodes[0], it still shows that the scene_dataset_config in VLNEpisode is set to 'default'. I paste the print result of print(habitat.get_config(config_path="benchmark/nav/vln_r2r.yaml"])) and print(env.episodes[0]), thanks for your reply!

 {'habitat': {'seed': 100, 'env_task': 'GymHabitatEnv', 'env_task_gym_dependencies': [], 'env_task_gym_id': '', 'environment': 
{'max_episode_steps': 500, 'max_episode_seconds': 10000000, 'iterator_options': {'cycle': True, 'shuffle': True, 'group_by_scene': 
True, 'num_episode_sample': -1, 'max_scene_repeat_episodes': -1, 'max_scene_repeat_steps': 10000, 'step_repetition_range': 0.2}}, 
'simulator': {'type': 'Sim-v0', 'forward_step_size': 0.25, 'turn_angle': 15, 'create_renderer': False, 'requires_textures': True, 'auto_sleep': 
False, 'step_physics': True, 'concur_render': False, 'needs_markers': True, 'update_articulated_agent': True, 'scene': 
'data/scene_datasets/habitat-test-scenes/van-gogh-room.glb', 'scene_dataset': 
'data/scene_datasets/mp3d/mp3d.scene_dataset_config.json', 'additional_object_paths': [], 'seed': '${habitat.seed}', 'default_agent_id': 
0, 'debug_render': False, 'debug_render_articulated_agent': False, 'kinematic_mode': False, 'should_setup_semantic_ids': True, 
'debug_render_goal': True, 'robot_joint_start_noise': 0.0, 'ctrl_freq': 120.0, 'ac_freq_ratio': 4, 'load_objs': True, 'hold_thresh': 0.15, 
'grasp_impulse': 10000.0, 'agents': {'main_agent': {'height': 1.5, 'radius': 0.1, 'max_climb': 0.2, 'max_slope': 45.0, 'grasp_managers': 1, 
'sim_sensors': {'rgb_sensor': {'type': 'HabitatSimEquirectangularRGBSensor', 'height': 640, 'width': 1280, 'position': [0.0, 0.88, 0.0], 
'orientation': [0.0, 0.0, 0.0]}, 'depth_sensor': {'type': 'HabitatSimEquirectangularDepthSensor', 'height': 241, 'width': 481, 'position': [0.0, 
0.88, 0.0], 'orientation': [0.0, 0.0, 0.0], 'min_depth': 0.0, 'max_depth': 10.0, 'normalize_depth': True}, 'semantic_sensor': {'type': 
'HabitatSimEquirectangularSemanticSensor', 'height': 640, 'width': 1280, 'position': [0.0, 0.88, 0.0], 'orientation': [0.0, 0.0, 0.0]}}, 
'is_set_start_state': False, 'start_position': [0.0, 0.0, 0.0], 'start_rotation': [0.0, 0.0, 0.0, 1.0], 'joint_start_noise': 0.1, 
'joint_start_override': None, 'articulated_agent_urdf': None, 'articulated_agent_type': None, 'ik_arm_urdf': None, 'motion_data_path': 
''}}, 'agents_order': ['main_agent'], 'default_agent_navmesh': True, 'navmesh_include_static_objects': False, 'habitat_sim_v0': 
{'gpu_device_id': 0, 'gpu_gpu': False, 'allow_sliding': True, 'frustum_culling': True, 'enable_physics': False, 'enable_hbao': False, 
'physics_config_file': './data/default.physics_config.json', 'leave_context_with_background_renderer': False, 'enable_gfx_replay_save': 
False}, 'ep_info': None, 'object_ids_start': 100, 'renderer': {'enable_batch_renderer': False, 'composite_files': None, 
'classic_replay_renderer': False}}, 'task': {'physics_target_sps': 60.0, 'reward_measure': None, 'success_measure': None, 
'success_reward': 2.5, 'slack_reward': -0.01, 'end_on_success': False, 'type': 'VLN-v0', 'lab_sensors': {'instruction_sensor': {'type': 
'InstructionSensor', 'instruction_sensor_uuid': 'instruction'}}, 'measurements': {'distance_to_goal': {'type': 'DistanceToGoal', 
'distance_to': 'POINT'}, 'success': {'type': 'Success', 'success_distance': 0.2}, 'spl': {'type': 'SPL'}}, 'rank0_env0_measure_names': 
['habitat_perf'], 'rank0_measure_names': [], 'goal_sensor_uuid': 'pointgoal', 'count_obj_collisions': True, 'settle_steps': 5, 
'constraint_violation_ends_episode': True, 'constraint_violation_drops_object': False, 'force_regenerate': False, 
'should_save_to_cache': False, 'object_in_hand_sample_prob': 0.167, 'min_start_distance': 3.0, 'render_target': True, 
'filter_colliding_states': True, 'num_spawn_attempts': 200, 'spawn_max_dist_to_obj': 2.0, 'base_angle_noise': 0.523599, 
'recep_place_shrink_factor': 0.8, 'ee_sample_factor': 0.2, 'ee_exclude_region': 0.0, 'base_noise': 0.05, 'spawn_region_scale': 0.2, 
'joint_max_impulse': -1.0, 'desired_resting_position': [0.5, 0.0, 1.0], 'use_marker_t': True, 'cache_robot_init': False, 'success_state': 
0.0, 'should_enforce_target_within_reach': False, 'task_spec_base_path': 'habitat/task/rearrange/pddl/', 'task_spec': '', 
'pddl_domain_def': 'replica_cad', 'obj_succ_thresh': 0.3, 'enable_safe_drop': False, 'art_succ_thresh': 0.15, 'robot_at_thresh': 2.0, 
'min_distance_start_agents': -1.0, 'actions': {'stop': {'type': 'StopAction'}, 'move_forward': {'type': 'MoveForwardAction', 'tilt_angle': 15}, 
'turn_left': {'type': 'TurnLeftAction', 'tilt_angle': 15}, 'turn_right': {'type': 'TurnRightAction', 'tilt_angle': 15}}}, 'dataset': {'type': 'R2RVLN-
v1', 'split': 'train', 'scenes_dir': 'data/scene_datasets/', 'content_scenes': ['*'], 'data_path': 
'data/datasets/vln/mp3d/r2r/v1/{split}/{split}.json.gz'}, 'gym': {'obs_keys': None, 'action_keys': None, 'achieved_goal_keys': [], 
'desired_goal_keys': []}}}

VLNEpisode(episode_id=1695, scene_id='data/scene_datasets/mp3d/rPc6DW4iMge/rPc6DW4iMge.glb', 
scene_dataset_config='default', additional_obj_config_paths=[], start_position=[-2.3039300441741943, 0.15942811965942383, 
-1.24167001247406], start_rotation=[-0.0, 0.9659258262890683, -0.0, -0.25881904510252063], info={'geodesic_distance': 
5.229569435119629}, _shortest_path_cache=<habitat_sim._ext.habitat_sim_bindings.MultiGoalShortestPath object at 
0x7fa235c34730>, goals=[NavigationGoal(position=[-3.637350082397461, 0.15942811965942383, -6.233019828796387], 
radius=3.0)], start_room=None, shortest_paths=None, reference_path=[[-2.3039300441741943, 0.15942811965942383, 
-1.24167001247406], [-2.188469886779785, 0.15942811965942383, -2.796689987182617], [-3.5894699096679688, 
0.15942811965942383, -3.483599901199341], [-3.586819887161255, 0.15942811965942383, -4.981349945068359], 
[-3.637350082397461, 0.15942811965942383, -6.233019828796387]], instruction=InstructionData(instruction_text='walk down hall 
toward bedroom, turn left at hallway, turn slight left into next hallway. turn slight right to bedroom, walk into bedroom and stop at 
dresser. ', instruction_tokens=[2584, 780, 1121, 2451, 273, 2494, 1360, 178, 1126, 2494, 2162, 1360, 1264, 1576, 1126, 15, 2494, 
2162, 1968, 2438, 273, 2584, 1264, 273, 119, 2278, 178, 797, 15]), trajectory_id=1080)