empty facial_transformation_matrixes

[DavidGasku]

Hi, having some fun with this promising project. I already have a "mask" projected onto the image, like a placeholder for makeups or similar. The result looks great, as every vertex is projected from the camera.

But for using 3D models, I need to get the transform from the face in world space. From what I understand, this can be acomplished by the FaceGeometry, which seems to be not implemented yet. But there is also a get_facial_transformation_matrixes method in the FaceLandmarkers, that should give that transformation. But it always returns empty.

So, is there anyway with the current state of the plugin to get that transformation? For now, I will try to estimate that transfomation, with several points in the face, and using the distance between the eyes as a constant.

And congratulations for your work, this could be huge for godot :)

[j20001970]

From mediapipe, facial_transformation_matrixes is a optional value that can be enabled from task optionsoutput_facial_transformation_matrixes which is false by default.

You could try setting output_facial_transformation_matrixes to true in MediaPipeFaceLandmarker to see if the method actually return any values.

[DavidGasku]

Thanks, it works! Not surprisingly, it's just the local transform, not the world one, and almost identical from the one I was already calculating using 3 points of the face.

Anyway, it is seems that the approach with the eye distance works well enough to estimate the distance to the camera.

[j20001970]

Since outputting local transform is the behaviour from mediapipe, I consider that this issue is solved.