One question about "batch_rigid_transform" function

[blairstar]

Thanks for your wonderful work. I have a question. In function batch_rigid_transform of lbs.py, what is the reason to do this: rel_transforms = transforms - F.pad(torch.matmul(transforms, joints_homogen), [3, 0, 0, 0, 0, 0, 0, 0]) ?

[Mamba-ZJP]

I also have the same question. Have you figured it out?

[you74674]

An equivalent expression:

R = transforms[..., :3, :3]
t = transforms[..., :3, 3:]
rel_transforms = transform_mat(R, t - R @ joints)
#note: dim in transform_mat should be -1, but you get the idea

But it's still really confusing. It just doesn't look correct for "The relative (with respect to the root joint) rigid transformations for all the joints".

[Joel18241096]

same question here

[Joel18241096]

If this is a mistake, does it mean the entire calculation process is wrong?

[xk-huang]

This minus term is equivalent to multiplying the inverse of the rest (T) pose. You can check the original formula in the SMPL paper. ps: I have also run into this problem when I started my project :) pps: the animation of SMPL is slightly different from that in the CG animation, while in the MANO model, they changed it back.

[you74674]

This minus term is equivalent to multiplying the inverse of the rest (T) pose.

If it's inverse, then shouldn't the rotation be transposed(inversed)? There is no any transpose or inverse in the code, and that's what get me confused.

[xk-huang]

The rotation part of the rest pose transformation matrix is identity transformation. Thus its inverse is equal to minus itself, i.e., (I|T)^(-1) = (I|-T).

[you74674]

So it's like this, and $t_{root}$ is equal to R @ joints?

$$ \begin{array}{l} T_{root}^{-1} \ T\ =\begin{bmatrix} R_{root} & t_{root}\ \end{bmatrix}^{-1} \ \begin{bmatrix} R & t\ \end{bmatrix}\ =\begin{bmatrix} R_{root}^{-1} & -R_{root}^{-1} t_{root}\ \end{bmatrix}\begin{bmatrix} R & t\ \end{bmatrix}\ =\begin{bmatrix} I & -t_{root}\ \end{bmatrix} \ \begin{bmatrix} R & t\ \end{bmatrix}\ =\begin{bmatrix} R & t-t_{root}\ \end{bmatrix} \end{array} $$

[AY-Liu]

It's not that complicated. Please don't think too much! It is just a simple translational transformation. Because you should all vertices based on thier original location, so this must add a translation. If you remove this, you will get antifact result.

[marluca]

Just because I stumbled across it and this should solve this:

$v^p$ denotes the posed vertex by joint j in world coordinates. Likewise, $v^r$ the vertex in world coordinates (not relative to the joint (yet)). Vertices are here in homogeneous coordinates. $T^{r^{-1}}$ is the inverse rest pose transformation that basically transforms the vertex in world coordinates into local coordinates of the unposed joint. $T^{p}$ is the posed transformation of joint j from local joint coordinates to world coordinates. The world coordinates of j are denoted as simply j.

v^p = T^p  T^{r^{-1}} v^r = T^p

\left(\begin{array}{cc} 
I & j\\
0 & 1
\end{array}\right)^{-1}
 v^r =

\left(\begin{array}{cc} 
R & t\\
0 & 1
\end{array}\right)
\left(\begin{array}{cc} 
I^T & -I^Tj\\
0 & 1
\end{array}\right)
v^r
= 
\left(\begin{array}{cc} 
R I^T & -R I^T j + t\\
0 & 1
\end{array}\right) v^r
=
\left(\begin{array}{cc} 
R & -R j + t\\
0 & 1
\end{array}\right) v^r

If you combine the transformations, you definitely see above that $T^{p}$ only needs to be adjusted in translation by the rotated joint's world coordinates, which is a bit unintuitive. That's why, I think of it as the following, where I denote $v^r_j$ as the vertex described in the local coordinates of the unposed joint.

v^p = T^p  T^{r^{-1}} v^r = T^p  v^r_j

[YogerChen]

An equivalent expression:

R = transforms[..., :3, :3]
t = transforms[..., :3, 3:]
rel_transforms = transform_mat(R, t - R @ joints)
#note: dim in transform_mat should be -1, but you get the idea

But it's still really confusing. It just doesn't look correct for "The relative (with respect to the root joint) rigid transformations for all the joints".

I think this code from @you74674 and the deduction from @marluca should be correct, but I was wondering whether their results are equivalent to the original code. The original code uses homogeneous coordinates for the joint positions and transforms is the 4*4 matrix with the translation term, so shouldn't F.pad(torch.matmul(transforms, joints_homogen), [3, 0, 0, 0, 0, 0, 0, 0]) gives us Rj+t, but not Rj we desire for in the subtraction from the transforms matrix?

[chaiyujin]

@YogerChen I also think the deduction from @marluca is correct. And I found the original code is equivalent to the result.

Note that the homogeneous coordinate for the joint is padded with zero: joints_homogen = F.pad(joints, [0, 0, 0, 1]), which acts as a vector (direction) rather than a position in homogeneous calculation. If the joint is $j$, its padded homogeneous coordinate is $\left( j \atop 0 \right)$.

With $T^p=\left( R \quad t \atop 0 \quad 1 \right)$: $$T^p \left( j \atop 0 \right) = \left( R \quad t \atop 0 \quad 1 \right) \left( j \atop 0\right)=\left( Rj \atop 0 \right)$$

Thus, the original code transforms - F.pad(torch.matmul(transforms, joints_homogen), [3, 0, 0, 0, 0, 0, 0, 0]) can be written as:

$$T^p-\left(0 \quad T^p \left( j \atop 0 \right)\right)=\left( R \quad t \atop 0 \quad 1 \right) - \left( 0 \quad Rj \atop 0 \quad 0 \right)=\left( R \quad t-Rj \atop 0 \quad\quad\quad 1 \right)$$

It's the same as the results from @marluca.