Question about face keypoints name

[buaacyw]

Thanks for your great work! I have a question about face key points names.

In human_models.py, it's mentioned that face key points names can be found in FLAME class. But I didn't find it. I have searched about FLAME face landmark orders but I can't find it. So could you please help me to find the FLAME face landmark orders that your model uses? (Like the below image)

Thanks！

[mks0601]

Those names are made by me, and actually FLAME does not provide any images like that.. So I just manually checked all landmarks.. One thing is that you can check here: https://github.com/mks0601/Hand4Whole_RELEASE/blob/Pose2Pose/data/MSCOCO/MSCOCO.py It provides compatible names with those of FLAME, so you can check it. You can check MSCOCO facial keypoints at https://github.com/jin-s13/COCO-WholeBody

[buaacyw]

Thanks for your quick reply! It really helps. So just to make sure: joint_to_flame[flame_landmark_index]=coco_index in which flame_landmark_index ranges from 0 to 72 (this is precisely the 73 total face landmark numbers mentioned in human_models.py). And except -1, coco_index range from 0 to 67. Add 24 to each coco_index, we can get the same index in the below image (which is from 24 to 91).

Is this right? Thanks!

[mks0601]

Maybe, but you may want to check it my yourself. It's been a while, so I can't remember clearly

[buaacyw]

Thanks!

[felixshing]

Hi guys, I would like ask a relevant question. Given that we can use a lot of keypoints on human's face, why we still resort to SMPL-X, which contains only 10 keypoints for facial expressions at the end?

[mks0601]

Could you elaborate? I have no idea what you mean by 10 keypoints. there are hundreads of mesh vertices in SMPL-X.

[felixshing]

Oh I mean SMPL-X has 10 parameters for controlling facial parameters. I thought they mean it has 10 keypoints on the face. Am I understanding it correctly?

What is the purpose for registering SMPL-X model here? Is it for evaluation or for generating human model?

I was wondering, since we can have a lot of keypoints, why we cannot just use them to generate pose but still resort to SMPL-X, which may lose some details (I am not sure whether such claim is true.) If we want our generated human model is photo-realistic, should we choose not to rely on SMPL-L or we can still use it?

[mks0601]

No. 10-dimensional vector represents a low-dimensional embedding of the facial expression like coeffiecient of the PCA space. keypoint is a subset of mesh vertices. each vertex of a mesh can be seen as a keypoint.

[felixshing]

Oh I see. So will adding some keypoints on face, such as using google' media pipe, help improve facial expressions when we use SMPL-X? And can we generate photo-realistic model with SMPL-X?