Visibility score in FaceLandmarker detection is always zero.

[ErikValle2]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

Ubuntu 22.04.4 LTS

Mobile device if the issue happens on mobile device

No response

Browser and version if the issue happens on browser

No response

Programming Language and version

Python

MediaPipe version

0.10.13

Bazel version

No response

Solution

Face Landmark detection

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

Visibility scores are always zero whether the face has occlusions or not. In the example, the first URL is a person wearing a mask, while the second is an image of Obama. Every single landmark shows a visibility equals to zero.

Describe the expected behaviour

I was expecting to have a value between 0 and 1

Standalone code/steps you may have used to try to get what you need

import cv2
import numpy as np

import mediapipe as mp
from mediapipe.tasks import python
from mediapipe.tasks.python import vision
import urllib.request

url="https://www.henryford.com/-/media/project/hfhs/henryford/henry-ford-blog/images/mobile-interior-banner-images/2020/07/mask-myths.jpg?h=600&iar=0&w=640&hash=BC25E61F0C97E5B83A27C93571FDCC57"
#url="https://upload.wikimedia.org/wikipedia/commons/8/8d/President_Barack_Obama.jpg"
url_response = urllib.request.urlopen(url)
img_array = np.array(bytearray(url_response.read()), dtype=np.uint8)
img = cv2.imdecode(img_array, -1)
frame = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
image = mp.Image(image_format=mp.ImageFormat.SRGB, data=frame)

# Create an FaceLandmarker object.
base_options = python.BaseOptions(model_asset_path='face_landmarker_v2_with_blendshapes.task')
options = vision.FaceLandmarkerOptions(base_options=base_options,
                                       output_face_blendshapes=True,
                                       output_facial_transformation_matrixes=True,
                                       num_faces=1)
detector = vision.FaceLandmarker.create_from_options(options)
detection_result = detector.detect(image)
any(t.visibility > 0 for t in detection_result.face_landmarks[0])

Other info / Complete Logs

WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1717157708.346293   21190 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
W0000 00:00:1717157708.346680   21190 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
/usr/local/lib/python3.11/site-packages/google/protobuf/symbol_database.py:55: UserWarning: SymbolDatabase.GetPrototype() is deprecated. Please use message_factory.GetMessageClass() instead. SymbolDatabase.GetPrototype() will be removed soon.
  warnings.warn('SymbolDatabase.GetPrototype() is deprecated. Please '

False

[matanox]

This is the current behavior as far as I know in all mediapipe landmark resulting pipelines. So this sounds more like an enhancement request, unless it's already a feature in which case I'm happy to learn about it.

[ErikValle2]

This is the current behavior as far as I know in all mediapipe landmark resulting pipelines. So this sounds more like an enhancement request, unless it's already a feature in which case I'm happy to learn about it.

I guess this feature already exists, but it might not be released to production yet.