ProHMR icon indicating copy to clipboard operation
ProHMR copied to clipboard

Wrong regression result with custom data.

Open xiezhy6 opened this issue 4 years ago • 2 comments

Hi, when I try to use ProHMR to test on my own data, the model fails to regress the correct pose, leading to a terrible fitting result. However, when I test another image with a similar pose, the model works well. (Success case and failure case are shown below.). My question is why the performance of ProHMR varies a lot with similar input images? And is there any solution to fix this issue (e.g., padding/cropping/resizing the input images, modifying the fitting code, etc.)?

Failure case: (from top to down: input image, regression result, fitting result) image

Success case: image

xiezhy6 avatar Dec 09 '21 12:12 xiezhy6

Hi, I would like to provide some of my test data, in which ProHMR does not work well. Maybe you could test them by yourself and help me analyze the reason for the failure cases. Thanks a lot! :-D id_13639849_img_0 id_13890083_img_0 id_14025181_img_0

xiezhy6 avatar Dec 09 '21 12:12 xiezhy6

Is there any specific reason the head of the people is cropped? Is it to preserve privacy? This could create a domain gap between your testing scenario and the images the network was trained on. In such a case you could try to finetuning ProHMR by simultating such types of truncations. Also does OpenPose work well on all those examples or not? I'm curious too see how the OpenPose keypoints look like.

nkolot avatar Dec 16 '21 02:12 nkolot