ShapeJac

[yang2640]

Hi Maria, thank you for your great work. I have a quick question on the Jacobian computation for the shape. In the E::MatrixXd SMPLWrapper::calcModel function: " if (shape_jac != nullptr) for (int i = 0; i < SMPLWrapper::SHAPE_SIZE; ++i) poseSMPL_(*pose, shape_jac[i], displacement); // Pose needs recalculation because joint positions at T are new " It seems "shape_jac" is mainly computed here, the logic seems to be a verts forward in poseSMPL, is this an approximation to the shape Jacobian ?

The shape Jacobian looks more complex ? It involves both "LBSMat" and "joints_global_transform" in "verts = LBSMat * joints_global_transform", I can't figure out how shape Jacobian is computed with regarding to the code.

[maria-korosteleva]

Hi @yang2640 Thank you for your interest in our work!

Shape jacobian is computed exactly here, following the analytical derivative of the SMPL model formula from the SMPL paper. The shape_jac[ that shapeSMPL_() outputs the derivative w.r.t. to each shape parameter when the model is in the rest pose (T-pose, pose vector is simply zero).

When the model is posed somehow, the shape jacobian needs to be updated to account for transformation due to pose, and that's what you see in the code snippet you mentioned.

Maybe, this simplified mathematical notation will help to get a better grasp of it. Note that the pose is constant when the shape derivative is computed. Shape blendshapes are always constant. Thus the model function is linear in each shape parameter, and derivative computation is straightforward.

model_verts(shape) =  pose_matrix * (shape_1 * shape_blendshape_1 + ... + shape_10 * shape_blendshape_10 + template_verts)

model_verts w.r.t. shape_i = pose_matrix * shape_blendshape_i

Mapping this to the code, shapeSMPL_() will output a collection of shape_blendshape_i as shape jacobian, poseSMPL_() will perform multiplication with the pose_matrix on the shape jacobian

Hope this helps =)

[yang2640]

Thanks for the reply @maria-korosteleva , but the shape will cause the change to shape blended vertices -> cause change to resting joint locations (joint regressor) -> cause change to global transform of lbs -> cause change to final vertices, so there should be some shape derivative on this chain ?

[maria-korosteleva]

@yang2640, If that's what you meant by "shape Jacobian looks more complex" in the original question, then yes, you are correct. I apologize for misunderstanding your original question.

You are right, the shape does influence the joint locations, and precise derivative should include that chain too. In this part, we chose to simplify the derivative formula by ignoring that influence

This seems reasonable as derivative is only valid locally around the shape parameters it's evaluated at. We expect that the influence on the final vertex positions due to the joints locations update is insignificant comparing to direct vertex update. We figured that the implementation of this chain won't worth it. Although we didn't make any formal comparisons.

This decision was made quite some time ago so that I even forgot I was following your train of thought myself =))