SMPLer-X icon indicating copy to clipboard operation
SMPLer-X copied to clipboard

Hand Rotation problem

Open liudongdong1 opened this issue 11 months ago • 7 comments

Hi, thanks for your great work again. I am utilizing the SMPL-X-H32 model to process an online video featuring extensive hand movements. When attempting to transform the model's hand rotation output into BVH motion files, I've observed that the resulting hand rotations are less than ideal. Despite adjusting the orientations, which improved some frames, inconsistencies persist in the hand orientations across others.

question 1. The degree of finger bending is too slight, and for some frame, the direction of finger bending seems wrong; image image question 2. i modify the direction of the above frame, it seems ok, but other frames seem wrong; image image question3. The facial parameters seem not taken effect. `class FaceRegressor(nn.Module): def init(self, feat_dim=768): super(FaceRegressor, self).init() self.expr_out = make_linear_layers([feat_dim, smpl_x.expr_code_dim], relu_final=False) self.jaw_pose_out = make_linear_layers([feat_dim, 6], relu_final=False)

def forward(self, expr_token, jaw_pose_token):
    expr_param = self.expr_out(expr_token)  # expression parameter
    jaw_pose = self.jaw_pose_out(jaw_pose_token)  # jaw pose parameter
    return expr_param, jaw_pose`

liudongdong1 avatar Mar 12 '24 08:03 liudongdong1

Hello, I am also studying hand Rotation problem at present, could you share the code related to your movement output? Thank you very much!

zxx123518 avatar Mar 15 '24 06:03 zxx123518

model hand's initial pose is different, use relaxing hand instead.

WingkitChou avatar Apr 01 '24 03:04 WingkitChou

Hi, the problem seems related to flat_hand_mean, smplerx assumes 'flat_hand_mean'=False. Please check this kwarg when creating the smplx model.

Wei-Chen-hub avatar Apr 03 '24 09:04 Wei-Chen-hub

I want to know if this problem was solved later? What is the reason?

TDIT-haha avatar May 31 '24 10:05 TDIT-haha

我想知道这个问题后来解决了吗?是什么原因?

后来我解决了问题,可以参考https://github.com/IDEA-Research/OSX/issues/84的问题解决方案,简单的解决方案是我在推理输出中将smplx的full_pose提取出来,这是有经过偏移处理的,再转fbx文件后手就没有问题了

TDIT-haha avatar Jun 01 '24 02:06 TDIT-haha

I want to know if this problem was solved later? What is the reason?

Yes, I was able to solve the problem using flat_hand_mean=False in kwarg while creating SMPLX model.

sonukiller avatar Jul 06 '24 08:07 sonukiller