Sensors configurations for reproducing baseline_ppo

[zhengzaiyi]

Hello, this paper gives training results with several sensors-setting at TABLE2. I want reproduce it on Gibson but I found that in ppo_pointnav.yaml there's an item:

SENSORS: ["DEPTH_SENSOR"]

And if I change it to "RGB_SENSOR", a connectionerror occurs. While in pointnav_gibson.yaml, I found another item

SENSORS: ["RGB_SENSOR"] # for agent_0

I'm confused and I don't know what the sensors actually mean since there's an apparent difference between them. How should I modify the configs to reproduce the paper's baseline results on RGB_SENSORS?

[erikwijmans]

The SENSORS field in the habitat-baselines config overrides the agent sensors. How much GPU memory do you have? The RGB sensor requires loading textures, so my guess is that you are running out of GPU memory and the simulator is crashing. For Gibson RGB, each instance of the sim needs ~1.5 GB of GPU memory.

[zhengzaiyi]

The SENSORS field in the habitat-baselines config overrides the agent sensors. How much GPU memory do you have? The RGB sensor requires loading textures, so my guess is that you are running out of GPU memory and the simulator is crashing. For Gibson RGB, each instance of the sim needs ~1.5 GB of GPU memory.

Thanks u so much. I’m using 2060s with 8G gpu-memory(I mistakenly thought it was a small job). So I guess there is a corresponding instance of sim for each python thread during the train?

[erikwijmans]

Yep, there is an instance of the sim in each process