learnable-triangulation-pytorch icon indicating copy to clipboard operation
learnable-triangulation-pytorch copied to clipboard

Problem with partial occluded bodies when evaluating on CMU dataset

Open Samleo8 opened this issue 4 years ago • 0 comments

This is an expansion of issue #75 where I was able to successfully test on the CMU Panoptic Studio dataset, with pretrained weights from Human3.6M dataset. When I use only camera views where the person's full body is seen, the results are quite satisfactory.

However, when done on cameras where some cameras show a full view, and some cameras have half of the body being occluded, the results are the same as the case if no cameras had a full view of the body: image

As you can see, possibly because of the occlusion, the model assumes that the feet are missing (but as @yurymalkov mentioned it's still close to the perceived ground). However, I believe that this should not be the case, as there are other views which should be able to tell the model where the feet actually is.

Why is this not the case? Is it possible that the model has not been exposed to such partial occlusions because of the limitations of the H36M dataset? Or is there some issue with the volumetric projections as a result of the faulty 2D heatmap prediction?

Thank you

Samleo8 avatar May 20 '20 16:05 Samleo8