The performance in the new scene

[cv1995]

In a new scene (no GT data, only camera parameters), is it possible to get accurate 3D poses by directly using the pretrained model you provided?

[jiahaoLjh]

Hi,

The pretrained model should work on new scenes as long as the range of depth is set correctly to cover the scene of interest.

[cv1995]

Hi,

The pretrained model should work on new scenes as long as the range of depth is set correctly to cover the scene of interest.

Thank you for your reply. How can I get the accurate depth range in the new scene? I use the synthesized 3D pose to calculate the approximate depth range according to our camera parameters and modify the POSE_MIN_ DEPTH and POSE_ MAX_ DEPTH(0.0-7000.0), but when using the Shelf pretrained model during inference, the following error results will appear(the 2D poses is the result of reprojection). What is the reason?

[jiahaoLjh]

You don't have to get the exact depth range. An approximate range that roughly covers the whole scene will work.

From you visualization, it seems the depth estimation is not correct. Can you visualize the following output data and check what is the depth distribution the network produces?

• pose_score_volume
• pose_depth_volume
• joint_score_volume
• joint_depth_volume

[cv1995]

You don't have to get the exact depth range. An approximate range that roughly covers the whole scene will work.

From you visualization, it seems the depth estimation is not correct. Can you visualize the following output data and check what is the depth distribution the network produces?

• pose_score_volume
• pose_depth_volume
• joint_score_volume
• joint_depth_volume

Can you explain in detail how to determine the depth range based on these four output datas? I find that as long as the depth range set during inference is inconsistent with that in training, errors will occur. Can I add your wechat for further communication?

[jiahaoLjh]

These data are the aggregated scores (pose_score_volume and joint_score_volume) and the output depth volume from the neural network indicating the likelihood of the depth being at a certain depth level (pose_depth_volume and joint_depth_volume). You can visualize these data to see (1) if the aggregated scores are showing reasonable cross-view correspondences and (2) if the output from the neural network is producing high responses at the correct depth level.

You can directly email me at [email protected] if you would like to have further discussion.

[AntonioEscamilla]

Hi. @cv1995 Did you manage to get better results for the 3d predictions? I am also having the same errors when using the provided pre-trained model in a multi-view scene of my own. The first thing I did was modify the depth range according to my scene, but the bad predictions continue regardless of whether I use a custom range or the same one used to train the model.

Thank you

[stevechaw]

Hi, Antonio, I want get the best 3d pose prediction. I look into the code in shelf.py, it seems has many 3d pose candidates, and compared to the GT to get the best candidate. How to get the best candidate when I don't have GT?