Pitch angle destorys the semantic map

[sycz00]

Hey folks,

I use the pr2 robot with your mapping examples and found out that the additional camera tilt joint causes the map to get destroyed. Is there anything I can do to overcome this issue ?

[devendrachaplot]

Hi,

You can try replacing 0 by the pitch angle in this line: https://github.com/devendrachaplot/Object-Goal-Navigation/blob/master/model.py#L185

[sycz00]

@devendrachaplot Hi Devendra,

thank you for your reply, honestly :). Now as I take the camera elevation angle into account, the map looks at least correctly segmented. Another issue is that as soon as the agent's pitch angle changes up or down, it assigns floor and ceilings to the obstacle map. Can I somehow control that via some thresholds or would it be the best choice to segment ceilings and floors in my segmentation mask to kinda "overdraw" them ?

Best, Fabian

[devendrachaplot]

Hi Fabian,

You probably need to change to camera_height. The camera_height needs to be set in meters using --camera_height argument. If your depth sensor is noisy, you can try increasing the --map_pred_threshold, --exp_pred_threshold and --cat_pred_threshold.

[sycz00]

Hi Devendra,

thanks again for your reply and sorry that I respond that late. Sadly it did not work out as expected, but I could remove everything except the walls and the objects I want to segment out of the depth map (assign 0.0 to them) and it seems to work. I do have another issue which occurs when the agent is moving through the map. The position of the agent seems to be constantly offset. What I mean is, while the agent move through the environment and segments some objects from one side, then turns around and segments the objects from another perspective. The objects including walls and semantic categories are getting segmented again on another position. Do you know what could cause this issue ? I am using the iGibson simulator and the way the pose and yaw angle of the agent are returned work the same.

I really tried to understand your code in detail, but I've had a hard time understanding a couple of lines.

1. The first code part which I do not understand is https://github.com/devendrachaplot/Object-Goal-Navigation/blob/master/envs/habitat/objectgoal_env.py#L449-L455. Why are you converting the agent angle with such if statements ?
2. Another I cannot really follow is this https://github.com/devendrachaplot/Object-Goal-Navigation/blob/master/model.py#L247-L271 especially the 57.29... hard coded values and also the switch between radians and degrees.
3. The last code snippet: https://github.com/devendrachaplot/Object-Goal-Navigation/blob/5d76902fe9be821926a1de32557ca9a8dc21d0f5/envs/utils/pose.py#L11-L21 Why are you taking this instead of just the difference pos2 - pos1 ? this differs in some cases from the ground truth in my case.

if you could give me any hints or source of information or even a tiny explanation, I would be so grateful. Best, Fabian

[devendrachaplot]

1. Angles are converted from quaternion to euler and to be within -pi to pi range.
2. 57.29 is used for conversion between radians and degrees.
3. Because we need relative pose between 2 poses not their difference.

Unfortunately, I can't help you with the pose logic in the iGibson simulator as I have not used it.