regression_prior

[listen-wind]

When I don't use regression_prior: combined， effect is very bad.How can i solve it.

[xiyichen]

Thanks for your interest in our work. As suggested in our report, using a more accurate pose prior is crucial for the performance improvement. The combined pose prior produces the best scores on the cropped-EHF benchmark. If you do not use the combined pose prior, I would still recommend to use the pose predictions from a single regression-based method, i.e., either 'ExPose' or 'PIXIE'.

[listen-wind]

Because, i want to estimate some image, but expose or other regression-baseed method do not get good result, so i can't use them.

When i do not use combined pose prior, below, two images are my result(stage_4) and keypoints image(by my own network).

So, Is the result normal? How can i improve it?

thank you!

[xiyichen]

Generally, our method relies on a reasonable body pose prediction by the regression-based methods as a more accurate body pose prior. If you do not use such a prior, then our method is equivalent to vanilla smplify-x + keypoints blending, which should still give slightly better results than original smplify-x.

If you do not use keypoint blending or pose prior from any regression-based methods, then you won't get any benefit from our tuned weights and initializations, and I would instead just recommend you to use the original smplify-x pipeline for fitting. (The body is also fully observed in your input image so it isn't well-suited for our modified pipeline designed for partially-observed human images).

If you use keypoints blending and/or pose prior, I would recommend modifying 'init_joints_idxs' here to incorporate also the lower-body joints in the optimization objective since these are observed in your input image. As an alternative to ExPose/PIXIE, you could considering using more recent human mesh recovery methods, such as OSX and SMPLer-X, to obtain more accurate body pose priors. However, it would require modifying the loading and initialization code here and here.

[listen-wind]

Because，i need estimate some half body images, so it's not appropriate to use original smplify-x pipeline.

I change the init_joints_idxs, hand_joints_weights and face_joints_weights(set the weight of hand and face to zero), got a better result, But right leg is a bit bad.

below, i remove four keypoints (tiptoes), i got a right pose with right leg,but tiptoe is a little high after fitting.

How can i solve it?

Thank you!

[xiyichen]

I'm not entirely sure but my guess is that the issue is with the keypoint weighting, as the detected locations look mostly reasonable. How are the confidence scores defined in your keypoint detector? Unfortunately I haven't met such an issue when I used openpose/mmpose/blended keypoints. You might want to blend your current keypoints with openpose/mmpose detections using the provided notebook, or manually tune the joint weights here.

[listen-wind]

I only use our pose detection. I don't set confidence scores(use default). The keypoint weighting is one when this keypoint in image, otherwise, is zero. Is it right?