How change graph vertical_fov_degrees value with input_side_packet or other way [python]

[tensorware]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Ubuntu 22.04.4

Task name (e.g. Image classification, Gesture recognition etc.)

face_landmarker_v2_with_blendshapes.task

Programming Language and version (e.g. C++, Python, Java)

Python

Describe the actual behavior

vision.FaceLandmarker.create_from_options(options) uses default environment for perspective_camera

Describe the expected behaviour

vision.FaceLandmarker.create_from_options(options) should allow side_inputs for perspective_camera

Standalone code/steps you may have used to try to get what you need

This is more or less the same question as in #3953, but by using the newer Task Solution API via Python. I'm using Face Landmarks Detection with MediaPipe Tasks similar to this example but with video live stream.

I want to improve the results (depth estimations) as mentioned here #3402, by setting camera parameters. See also: https://github.com/google/mediapipe/blob/72e4d3ee61953af2295fbb958c4ee32d27489d95/mediapipe/tasks/cc/vision/face_landmarker/face_landmarker_graph.cc#L207

How this can be done with MediaPipe v0.10.11 python bindings, without re-building from source?

Other info / Complete Logs

To summarize:

Is there a way to change vertical_fov_degrees using the latest python bindings,

# STEP 1: Import the necessary modules.
import mediapipe as mp
from mediapipe.tasks import python
from mediapipe.tasks.python import vision

# STEP 2: Create an FaceLandmarker object.
base_options = python.BaseOptions(model_asset_path='face_landmarker_v2_with_blendshapes.task')
options = vision.FaceLandmarkerOptions(base_options=base_options,
                                       output_face_blendshapes=True,
                                       output_facial_transformation_matrixes=True,
                                       num_faces=1)
detector = vision.FaceLandmarker.create_from_options(options)

by injecting side packets (or otherwise) before or after the detector instantiation?

I traced it down to be somewhere here: https://github.com/google/mediapipe/blob/72e4d3ee61953af2295fbb958c4ee32d27489d95/mediapipe/tasks/python/vision/face_landmarker.py#L3104

Any suggestions are welcome, thanks.

[kuaashish]

Hi @tensorware,

Following our internal discussion, please be informed that at present, accomplishing this task requires rebuilding from the source. However, we acknowledge that simplifying this process is a priority for us in the near future.

Thank you!!

[tensorware]

Many thanks for the answer. In the meantime, is there any other way to change the graph config or the task file?

• I have seen that the graph config can be accessed from the detector. Is there a way to update the config object before inference?

# STEP 1: Import the necessary modules.
import mediapipe as mp
from mediapipe.tasks import python
from mediapipe.tasks.python import vision

# STEP 2: Create an FaceLandmarker object.
base_options = python.BaseOptions(model_asset_path='face_landmarker_v2_with_blendshapes.task')
options = vision.FaceLandmarkerOptions(base_options=base_options,
                                       output_face_blendshapes=True,
                                       output_facial_transformation_matrixes=True,
                                       num_faces=1)
detector = vision.FaceLandmarker.create_from_options(options)

with open('graph.pbtxt', 'w') as file:
    config = detector.get_graph_config() # Update this config ?
    file.write(str(config))

• Would it be possible to create a custom face_landmarker_v2_with_blendshapes.task file, instead of re-building everything from source?

[kuaashish]

Hi @tensorware,

Apologies for the delay in responding. After receiving additional input from our team, It is currently not possible. The only remaining option is to rebuild from the source by altering the graph.

Thank you!!

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No