What is the ground truth we used for training semantic global policy?

[rginpan]

Does anybody can give me some explanations about:

• I think the semantic mapping module is not a learnable module, am I right?
• What is the ground truth we need to train the semantic global policy, the top-view 2D semantic map?

Thanks!!

[FUIGUIMURONG]

I am reading this paper and want to do the experiment. For the first question: As it says in the paper, the semantic mapping module should be trained. The Mask RCNN weights are frozen but the denoising network should be trained. For the second question: As it says in the paper, The Goal-Oriented Semantic Policy decides a long-term goal based on the current semantic map to reach the given object goal. It takes the semantic map, the agent’s current and past locations, and the object goal as input and predicts a long-term goal in the top-down map space.