How to scan a scene?

[HLinChen]

Habitat-Sim version

v0.2.2

Docs and Tutorials

Did you read the docs? https://aihabitat.org/docs/habitat-sim/ Yes

Did you check out the tutorials? https://aihabitat.org/tutorial/2020/ Yes

❓ Questions and Help

I want to extract images by image extractor. But it needs camera poses. The existing method provided by habitat-sim is 'closest_point_extractor'. But it generates camera positions randomly, which don't cover the whole scene and are not in a smooth trajectory. Is there any way to generate a smooth trajectory to scan the whole scene using habitat-sim?

[aclegg3]

Hey @HLinChen,

Sounds like you want a method for generating a set of images which maximize coverage of the scene content. Unfortunately, this is an open problem. I can propose a couple of heuristics, but it would be up to you to choose and implement a solution.

My thoughts:

1. Maybe you are interested in views which a human or agent would get while navigating the scene. In this case, maybe you could compute a set of waypoints on the navmesh and use the ShortestPath module to connect them. Camera orientations could be oscillatory (left -> right -> left) or scripted somehow.
2. Maybe you want to capture 360 images from a uniformly distributed set of points in free-space. The sensor module provides spherical and cube map sensors.
3. You could manually fly a camera through the scenes.

[dhruvbatra]

For MP3D, you can find some trajectories here: https://github.com/vincentcartillier/Semantic-MapNet

[HLinChen]

Thanks for your suggestions! I will try it.