FaceLandmarker's iris landmarks are worse compared to FaceMesh(refine_landmarks=True)

[twoertwein]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

No

OS Platform and Distribution

Mac M2

MediaPipe Tasks SDK version

0.10.10

Task name (e.g. Image classification, Gesture recognition etc.)

FaceLandmarker

Programming Language and version (e.g. C++, Python, Java)

Python

Describe the actual behavior

Iris landmarks are inaccurate

Describe the expected behaviour

At least as good/better than with FaceMesh(refine_landmarks=True)

Standalone code/steps you may have used to try to get what you need

Call FaceLandmarker and FaceMesh(refine_landmarks=True)

Other info / Complete Logs

The attention mesh model of (the legacy) FaceMesh(refine_landmarks=True) produces good iris landmarks! When switching to FaceLandmarker, the iris landmarks seem to be far worse compared to the FaceMesh: qualitatively worse but also quantitatively worse when using the landmarks to derive eye gaze and then comparing it with OpenFace.

Since the use of FaceMesh is discouraged, it would be great if its replacement (FaceLandmarker) would be on par with it, including for iris landmarks.

I'm not familiar with TensorFlow Lite files. As the model files for both approaches are available - is it possible to manually replace the eye model in FaceLandmarker?

[twoertwein]

I think I was too quick to blame FaceLandmarker - eye gaze based on eye landmarks is just very noisy. On other videos, FaceLandmarker performs similar/better

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[twoertwein]

It seems that FaceLandmarker is mostly worse for lower resolution images but FaceMesh still performs reasonably well on them.

[kuaashish]

Hi @twoertwein,

Yes, We agree with your observations. However, as we no longer maintain legacy solutions, we are unable to provide much support for them. For now, our current focus is on continuously improving our Task API.

Thank you!!

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No