Some confusion about the sequence of scene-alignment and SMPL optimization

[yijicheng]

Thanks for your excellent work!

In the paper, SMPL optimization is done before scene-alignment. However, the sequence is reserved in the code. Which one is correct?

"Therefore, we refine the SMPL estimates by optimizing the SMPL parameters using silhouette estimated from [8,48], and 2D joints estimated from [3,4] as detailed in the supplementary material. We then align the SMPL estimates in the scene coordinates."

At the same time, I found the head of SMPL is smaller than G.T. in my own data. Is the mismatching caused by the reversed sequence?

[jiangwei221]

Hi @yijicheng, in practice we first do the SMPL-scene alignment, then the refinement, as in the code. I think the misalignment in the image is mainly caused by the global orientation of the human, i.e. the SMPL mesh is facing a slightly wrong direction than the GT. I suggest you could try:

1. Visualize and check if the foot are touching the ground plane.
2. Train the human NeRF models, and let the RGB supervision to correct the poses in this case.

[yijicheng]

Hi @yijicheng, in practice we first do the SMPL-scene alignment, then the refinement, as in the code. I think the misalignment in the image is mainly caused by the global orientation of the human, i.e. the SMPL mesh is facing a slightly wrong direction than the GT. I suggest you could try:

1. Visualize and check if the foot are touching the ground plane.
2. Train the human NeRF models, and let the RGB supervision to correct the poses in this case.

Thank you so much, I will try your suggestions!

[jiangwei221]

Btw, the refinement in preprocessing only taking care of the limbs, it won't correct the global orientation.

[yijicheng]

Btw, the refinement in preprocessing only taking care of the limbs, it won't correct the global orientation.

got it, I see you only keep the grad of limbs in the optimization.