mediapipe
mediapipe copied to clipboard
google-ai-edge
selfie_multiclass task with option delegate=BaseOptions.Delegate.GPU not working
Have I written custom code (as opposed to using a stock example script provided in MediaPipe)
None
OS Platform and Distribution
Ubuntu 22.04.3 LTS (docker image: tensorflow:2.16.1-gpu-jupyter)
MediaPipe Tasks SDK version
0.10.14
Task name (e.g. Image classification, Gesture recognition etc.)
selfie_multiclass
Programming Language and version (e.g. C++, Python, Java)
Python
Describe the actual behavior
delegate=BaseOptions.Delegate.GPU not work
Describe the expected behaviour
enable GPU support
Standalone code/steps you may have used to try to get what you need
import mediapipe as mp
import numpy as np
import cv2
model_path = '/models/mediapipe/selfie_multiclass_256x256.tflite'
IMAGE_FILE = 'input.jpg'
BaseOptions = mp.tasks.BaseOptions
ImageSegmenter = mp.tasks.vision.ImageSegmenter
ImageSegmenterOptions = mp.tasks.vision.ImageSegmenterOptions
VisionRunningMode = mp.tasks.vision.RunningMode
# ImageSegmenter
options = ImageSegmenterOptions(
#base_options=BaseOptions(model_asset_path=model_path),
base_options = BaseOptions(model_asset_path=model_path, delegate=BaseOptions.Delegate.GPU),
running_mode=VisionRunningMode.IMAGE,
output_category_mask=True)
colors = [
[255, 0, 0], # background: blue
[255, 255, 0], # hair: skyblue
[0, 255, 0], # body-skin: green
[255, 0, 128], # face-skin: purple
[255, 0, 255], # clothes: magenta
[0, 128, 255] # others: orange
]
category = ["background", "hair", "body-skin", "face-skin", "clothes", "others"]
with ImageSegmenter.create_from_options(options) as segmenter:
mp_image = mp.Image.create_from_file(IMAGE_FILE)
original_image = mp_image.numpy_view()
# original_image = original_image[:, :, :3] # remove alpha channel
# masks
segmented_masks = segmenter.segment(mp_image)
# category_mask
category_mask = segmented_masks.category_mask
category_mask_np = category_mask.numpy_view()
h, w = category_mask_np.shape # numpy width, height
color_image = np.zeros((h, w, 3), dtype=np.uint8) #generate same size empty image
for i, color in enumerate(colors): # fill every class
color_image[category_mask_np == i] = color
alpha = 0.5
blended_image = cv2.addWeighted(original_image, 1 - alpha, color_image, alpha, 0)
status = cv2.imwrite('path_to_save_image.jpg', blended_image)
print("Image written to file-system: ", status)
Other info / Complete Logs
When I set base_options=BaseOptions(model_asset_path=model_path), the code worked, when I set base_options = BaseOptions(model_asset_path=model_path, delegate=BaseOptions.Delegate.GPU), the code got error:
'''
2024-06-25 02:26:50.945231: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1719282412.338260 551 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
Traceback (most recent call last):
File "/tools/mask_1_img.py", line 37, in <module>
with ImageSegmenter.create_from_options(options) as segmenter:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/mediapipe/tasks/python/vision/image_segmenter.py", line 268, in create_from_options
return cls(
^^^^
File "/usr/local/lib/python3.11/dist-packages/mediapipe/tasks/python/vision/image_segmenter.py", line 145, in __init__
super(ImageSegmenter, self).__init__(
File "/usr/local/lib/python3.11/dist-packages/mediapipe/tasks/python/vision/core/base_vision_task_api.py", line 70, in __init__
self._runner = _TaskRunner.create(graph_config, packet_callback)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Service "kGpuService", required by node mediapipe_tasks_vision_image_segmenter_imagesegmentergraph__mediapipe_tasks_core_inferencesubgraph__inferencecalculator__mediapipe_tasks_vision_image_segmenter_imagesegmentergraph__mediapipe_tasks_core_inferencesubgraph__InferenceCalculator, was not provided and cannot be created: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
'''
run the script in tensorflow:2.16.1-gpu-jupyter container
import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))
'''
>>> import tensorflow as tf
2024-06-25 03:53:58.671721: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
>>> print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))
2024-06-25 03:54:00.062634: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2024-06-25 03:54:00.067000: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2024-06-25 03:54:00.067152: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
Num GPUs Available: 1
'''
Hi @SoaringTiger,
Thank you for bringing this to our attention again. It appears that our multi-class selfie segmentation currently does not support GPU usage. A similar issue was reported previously at https://github.com/google-ai-edge/mediapipe/issues/5081. Please allow us some time to investigate further, and I will provide an update soon.
Thank you!!
Hi @SoaringTiger,
Could you confirm if you can run other examples in your GPU environment, such as the Hand example provided here?
Thank you!!
Hi @SoaringTiger,
Could you confirm if you can run other examples in your GPU environment, such as the Hand example provided here?
Thank you!!
Hi @kuaashish
# STEP 2: Create an HandLandmarker object.
base_options = python.BaseOptions(
model_asset_path='/models/mediapipe/hand_landmarker.task',
#delegate=BaseOptions.Delegate.GPU
)
options = vision.HandLandmarkerOptions(base_options=base_options,
num_hands=2)
detector = vision.HandLandmarker.create_from_options(options)
When I run the code without GPU, got warning, but I can get result image:
I0000 00:00:1720067300.162163 406 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe[/gpu/gl_context_egl.cc:77](http://192.168.2.8:8888/gpu/gl_context_egl.cc#line=76)) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
W0000 00:00:1720067300.170902 948 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.
W0000 00:00:1720067300.186529 948 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.
When I run the code with 'delegate=BaseOptions.Delegate.GPU', I got error:
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1720067775.707569 1064 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
Traceback (most recent call last):
File "/sam/mp_mask_1_hands.py", line 65, in <module>
detector = vision.HandLandmarker.create_from_options(options)
File "/usr/local/lib/python3.10/dist-packages/mediapipe/tasks/python/vision/hand_landmarker.py", line 364, in create_from_options
return cls(
File "/usr/local/lib/python3.10/dist-packages/mediapipe/tasks/python/vision/core/base_vision_task_api.py", line 70, in __init__
self._runner = _TaskRunner.create(graph_config, packet_callback)
RuntimeError: Service "kGpuService", required by node mediapipe_tasks_vision_hand_landmarker_handlandmarkergraph__mediapipe_tasks_vision_hand_detector_handdetectorgraph__mediapipe_tasks_core_inferencesubgraph__inferencecalculator__mediapipe_tasks_vision_hand_landmarker_handlandmarkergraph__mediapipe_tasks_vision_hand_detector_handdetectorgraph__mediapipe_tasks_core_inferencesubgraph__InferenceCalculator, was not provided and cannot be created: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
Hi @SoaringTiger,
It seems that some drivers are missing, preventing the GPU from being detected in the environment. Can you please try this Colab for benchmarking some of the tasks: Colab Notebook? Additionally, could you adopt the same for your environment and update us on the status?
Thank you!!
Hi @SoaringTiger,
It seems that some drivers are missing, preventing the GPU from being detected in the environment. Can you please try this Colab for benchmarking some of the tasks: Colab Notebook? Additionally, could you adopt the same for your environment and update us on the status?
Thank you!!
Hi, @kuaashish my nvidia-smi info in docker container, docker image is tensorflow/tensorflow:2.16.2-gpu-jupyter +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 545.29.06 Driver Version: 545.29.06 CUDA Version: 12.3 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA GeForce RTX 4060 Ti Off | 00000000:01:00.0 On | N/A | | 0% 52C P8 8W / 165W | 14558MiB / 16380MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+
I have install the mediapipe-nightly and I can only run mediapipe task script without gpu option: colab
I have tested tensorflow with gpu in container
import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))
matrix1 = tf.random.uniform([1000, 1000])
matrix2 = tf.random.uniform([1000, 1000])
with tf.device('/GPU:0'):
product = tf.matmul(matrix1, matrix2)
with tf.device('/GPU:0'):
print("Matrix multiplication result on GPU:")
print(product)
Got result
Num GPUs Available: 1
Matrix multiplication result on GPU:
tf.Tensor(
[[243.63484 238.3081 250.76256 ... 249.20293 241.33276 251.67079]
[249.44817 234.74933 245.9238 ... 242.76912 239.75523 247.66855]
[238.73503 231.70839 242.5265 ... 243.06746 235.40015 246.4232 ]
...
[236.47717 229.38445 241.08224 ... 236.93318 233.91486 245.26237]
[246.1244 242.03398 246.94525 ... 250.89856 238.61884 247.93216]
[248.92546 240.55031 253.50462 ... 248.77428 250.05797 252.9958 ]], shape=(1000, 1000), dtype=float32)
Hi @kinarr,
Could you please assist with resolving the issue of not being able to utilize the GPU in the Docker environment?
Thank you!!
I have exactly the same setup and erros as this issue, but using FaceLandmarker. Looking forward to hearing back from you guys.
Error prompt:
File "/usr/local/lib/python3.11/dist-packages/mediapipe/tasks/python/vision/face_landmarker.py", line 3104, in create_from_options
return cls(
^^^^
File "/usr/local/lib/python3.11/dist-packages/mediapipe/tasks/python/vision/core/base_vision_task_api.py", line 70, in __init__
self._runner = _TaskRunner.create(graph_config, packet_callback)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Service "kGpuService", required by node mediapipe_tasks_vision_face_landmarker_facelandmarkergraph__mediapipe_tasks_vision_face_detector_facedetectorgraph__mediapipe_tasks_core_inferencesubgraph__inferencecalculator__mediapipe_tasks_vision_face_landmarker_facelandmarkergraph__mediapipe_tasks_vision_face_detector_facedetectorgraph__mediapipe_tasks_core_inferencesubgraph__InferenceCalculator, was not provided and cannot be created: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:84) egl_initializedUnable to initialize EGL
I have exactly the same setup and erros as this issue, but using FaceLandmarker. Looking forward to hearing back from you guys.
Error prompt:
File "/usr/local/lib/python3.11/dist-packages/mediapipe/tasks/python/vision/face_landmarker.py", line 3104, in create_from_options return cls( ^^^^ File "/usr/local/lib/python3.11/dist-packages/mediapipe/tasks/python/vision/core/base_vision_task_api.py", line 70, in __init__ self._runner = _TaskRunner.create(graph_config, packet_callback) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: Service "kGpuService", required by node mediapipe_tasks_vision_face_landmarker_facelandmarkergraph__mediapipe_tasks_vision_face_detector_facedetectorgraph__mediapipe_tasks_core_inferencesubgraph__inferencecalculator__mediapipe_tasks_vision_face_landmarker_facelandmarkergraph__mediapipe_tasks_vision_face_detector_facedetectorgraph__mediapipe_tasks_core_inferencesubgraph__InferenceCalculator, was not provided and cannot be created: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:84) egl_initializedUnable to initialize EGL
About FaceLandmarker:
Face blendshape model contains CPU only ops. Sets FaceBlendshapesGraph acceleration to Xnnpack.
It means not all models were modified to support GPU processing. In this case, the documentation mentions that only the face detector (BlazeFace short-range) bundled to FaceLandmarker is optimized for GPU inference. Therefore, I recommend you to check the model cards first to verify if it does support GPU.