Question about loading semantic annotation in HM3D Dataset

[maengjemo]

Habitat-Sim version

vx.x.x

Habitat is under active development, and we advise users to restrict themselves to stable releases. Are you using the latest release version of Habitat-Sim? Your question may already be addressed in the latest version. We may also not be able to help with problems in earlier versions because they sometimes lack the more verbose logging needed for debugging.

Main branch contains 'bleeding edge' code and should be used at your own risk.

Docs and Tutorials

Did you read the docs? https://aihabitat.org/docs/habitat-sim/ yes Did you check out the tutorials? https://aihabitat.org/tutorial/2020/ yes Perhaps your question is answered there. If not, carry on!

❓ Questions and Help

I wonder is there any problem loading semantic annotation this way.

According to the example of MP3D dataset https://colab.research.google.com/github/facebookresearch/habitat-sim/blob/v0.1.6/examples/tutorials/colabs/ECCV_2020_Navigation.ipynb

I loaded config file habitat-lab/data/scene_datasets/hm3d/hm3d_annotated_basis.scene_dataset_config.json and chose test scene habitat-lab/data/scene_datasets/hm3d/minival/00800-TEEsavR23oF/TEEsavR23oF.basis.glb

The output of function print_scene_recur returns House has 0 levels, 14 regions and 661 objects House center:[0. 0. 0.] dims:[-inf -inf -inf] and all of the object's names were Unknown_{int}. It assumed scene annotation is not loaded well.

The tutorial MP3D example was worked well. Is there any point I missed?

-- here is full code of my current state

import habitat_sim
backend_cfg = habitat_sim.SimulatorConfiguration()
backend_cfg.scene_id = "/data/scene_datasets/hm3d/minival/00800-TEEsavR23oF/TEEsavR23oF.basis.glb"
backend_cfg.scene_dataset_config_file = "/data/scene_datasets/hm3d/minival/hm3d_annotated_minival_basis.scene_dataset_config.json"

sem_cfg = habitat_sim.CameraSensorSpec()
sem_cfg.uuid = "semantic"
sem_cfg.sensor_type = habitat_sim.SensorType.SEMANTIC

agent_cfg = habitat_sim.agent.AgentConfiguration()
agent_cfg.action_space = {
    "move_forward": habitat_sim.agent.ActionSpec(
        "move_forward", habitat_sim.agent.ActuationSpec(amount=0.25)
    ),
    "turn_left": habitat_sim.agent.ActionSpec(
        "turn_left", habitat_sim.agent.ActuationSpec(amount=30.0)
    ),
    "turn_right": habitat_sim.agent.ActionSpec(
        "turn_right", habitat_sim.agent.ActuationSpec(amount=30.0)
    ),
}
agent_cfg.sensor_specifications = [sem_cfg]

sim_cfg = habitat_sim.Configuration(backend_cfg, [agent_cfg])
sim = habitat_sim.Simulator(sim_cfg)

def print_scene_recur(scene, limit_output=10):
    print(f"House has {len(scene.levels)} levels, {len(scene.regions)} regions and {len(scene.objects)} objects")
    print(f"House center:{scene.aabb.center} dims:{scene.aabb.sizes}")
    count = 0
    for level in scene.levels:
        print(
            f"Level id:{level.id}, center:{level.aabb.center},"
            f" dims:{level.aabb.sizes}"
        )
        for region in level.regions:
            print(
                f"Region id:{region.id}, category:{region.category.name()},"
                f" center:{region.aabb.center}, dims:{region.aabb.sizes}"
            )
            for obj in region.objects:
                print(
                    f"Object id:{obj.id}, category:{obj.category.name()},"
                    f" center:{obj.aabb.center}, dims:{obj.aabb.sizes}"
                )
                count += 1
                if count >= limit_output:
                    return None
scene = sim.semantic_scene
print_scene_recur(scene)

[aclegg3]

Hey @jemo-mh This is working for me locally. I am seeing a large number of unknown colorings for this scene, but I believe they are isolated vertex groups missed by the paint. I've attached a file TEEsavR23oF.basis_CC_report.csv which shows the semantic report of the scene.

To get this I ran the viewer.cpp application like:

./build/viewer --dataset data/scene_datasets/hm3d/minival/hm3d_annotated_minival_basis.scene_dataset_config.json -- data/scene_datasets/hm3d/minival/00800-TEEsavR23oF/TEEsavR23oF.basis.glb

and then pressed the 'j' key to generate the report. You should be able to repeat this locally and get the same result.

Edit: if you don't get this result we can dig further into what the issue may be. As a related note, you should see a directory structure for the scene like:

[ZetaRing]

Same problem for me. Pls post here if any luck.