An error occurs when executing the "object_detector" routine after executing the "mediapipe lanmdmark" routine

[kn2234]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

windows10

Mobile device if the issue happens on mobile device

none

Browser and version if the issue happens on browser

none

Programming Language and version

python3.11

MediaPipe version

0.10.2

Bazel version

none

Solution

none

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

none

Xcode & Tulsi version (if issue is related to building for iOS)

none

Describe the actual behavior

If the code in ① below (I narrowed it down to the code directly linked to the cause) is executed upstream of ②, the following error will occur during the execution of the code in ② below. If the code in ① below does not exist, there is no problem.
It seems that the setting used to execute object_detector is affected by the execution of ① landmark.
Please tell me how to estimate the cause and how to deal with it.
If the code in ① below (I narrowed it down to the code directly linked to the cause) is executed upstream of ②, the following error will occur during the execution of the code in ② below. If the code in ① below does not exist, there is no problem.
It seems that the setting used to execute object_detector is affected by the execution of ① landmark.
Please tell me how to estimate the cause and how to deal with it.

①with mp_pose.Pose(
         static_image_mode=True,
         model_complexity=2,
         min_detection_confidence=0.5) as pose:

②base_options = python.BaseOptions(model_asset_path='efficientdet.tflite')
         options = vision.ObjectDetectorOptions(base_options=base_options,
                                                score_threshold=0.3)
         detector = vision.ObjectDetector.create_from_options(options)

[error code]
Traceback (most recent call last):
   File "c:\Users\aaabbbccc\Documents\python\object_detector_test.py", line 301, in <module>
     detector = vision.ObjectDetector.create_from_options(options)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "C:\Users\aaabbbccc\Documents\python\venv\Lib\site-packages\mediapipe\tasks\python\vision\object_detector.py", line 234, in create_from_options
     return cls(
            ^^^^
   File "C:\Users\aaabbbccc\Documents\python\venv\Lib\site-packages\mediapipe\tasks\python\vision\core\base_vision_task_api.py", line 70, in __init__
     self._runner = _TaskRunner.create(graph_config, packet_callback)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Unable to open file at C:\Users\aaabbbccc\Documents\python\venv\Lib\site-packages/C:\Users\aaabbbccc\Documents\python\efficientdet.tflite, errno=22

Describe the expected behaviour

I want the code in ① below (I narrowed it down to the code that is directly linked to the cause) to work without problems even if it is executed upstream of ②.
Is it possible to reset the landmark settings made in ① so that they will not be affected by ②?

Standalone code/steps you may have used to try to get what you need

see above

Other info / Complete Logs

none

[kn2234]

Supplement. Judging from the error code, it seems to be related to the running mode, but this time, the image handled by ① is a frame image of the camera input, and ② is a single image file that is imported and processed.

[kuaashish]

@kn2234,

Could you elaborate your query with complete details? Also, Please share the complete instructions following from the documentation to reproduce the issue from our end. Thank you

[kn2234]

Paste the code you're having trouble with. When checking the reproduction, the local folder path to be called after writing the image file is set in two places for each object detection, so please set it individually and try it. Since it was created as a test to reproduce the phenomenon, the code is to operate the landmark after executing the first object detection, and then perform the same object detection again. If there is no problem, object detection will be repeated twice, but an error will occur at the second time as declared. Of course, as explained above, if you remove that code so that it doesn't run landmarks in the middle, it will run two object detections without issue. I would appreciate it if you could suggest the cause and how to deal with it thank you //////////////////////////////////////////////// import sys sys.path.append('venv\Lib\site-packages')

import os import time import datetime import pytz import cv2 import mediapipe as mp import matplotlib.pyplot as plt mp_drawing = mp.solutions.drawing_utils mp_drawing_styles = mp.solutions.drawing_styles mp_pose = mp.solutions.pose import tensorflow as tf import tensorflow_hub as hub import numpy as np from mediapipe.tasks import python from mediapipe.tasks.python import vision

import urllib.request url = "https://storage.googleapis.com/mediapipe-models/object_detector/efficientdet_lite0/int8/1/efficientdet_lite0.tflite" filename = "efficientdet.tflite" urllib.request.urlretrieve(url, filename)

import numpy as np MARGIN = 10 # pixels ROW_SIZE = 10 # pixels FONT_SIZE = 1 FONT_THICKNESS = 1 TEXT_COLOR = (255, 0, 0) # red

#Object detection visualization processing routine def visualize( image, detection_result ) -> np.ndarray: for detection in detection_result.detections: #draw bounding box bbox = detection.bounding_box start_point = bbox.origin_x, bbox.origin_y end_point = bbox.origin_x + bbox.width, bbox.origin_y + bbox.height cv2.rectangle(image, start_point, end_point, TEXT_COLOR, 3)

#Draw labels and scores
category = detection.categories[0]
category_name = category.category_name
probability = round(category.score, 2)
result_text = category_name + ' (' + str(probability) + ')'
text_location = (MARGIN + bbox.origin_x,
                 MARGIN + ROW_SIZE + bbox.origin_y)
cv2.putText(image, result_text, text_location, cv2.FONT_HERSHEY_PLAIN,
            FONT_SIZE, TEXT_COLOR, FONT_THICKNESS)

return image

#Create a capture to get video from a USB camera cap = cv2.VideoCapture(0)

#Check if the camera capture was successfully opened if not cap.isOpened(): print("The camera did not open successfully!!") sys.exit()

#flame fps = cap.get(cv2.CAP_PROP_FPS) print(fps) count = 0 #frame counter start_time = time.time() #Record start time while True: ret, frame = cap.read() if not ret: #Exit the loop if the load fails break
if int(count % fps/10) == 0: #Capture and save frame images every 0.1 seconds ((fps=30)/10) path_num = int(count // fps) frame = frame.copy() #Make a copy of the frame image ########################################### #Set the folder path as an individual save environment! ! cv2.imwrite(f"C:/Users/*Please write the folder path name!/detection_result/object_detector.jpg", frame) cv2.imshow("frame1", frame)

################################################ #Set the folder path as an individual save environment! ! #Call the saved image to move the object detection with a still image image_path = r"C:/Users/*Please write the folder path name!/detection_result/object_detector.jpg"
img = cv2.imread(image_path)
if img is None:
print("Failed to load image!!")

    cv2.imshow("Image", img)   
    cv2.waitKey(50)
      
    #Perform object detection and visualize results
    #Step 1: Create an ObjectDetector Object
    base_options = python.BaseOptions(model_asset_path='efficientdet.tflite')
    options = vision.ObjectDetectorOptions(base_options=base_options,
                                           score_threshold=0.5)     
    detector = vision.ObjectDetector.create_from_options(options)

    #Step 2: Load Input Image
    image = mp.Image.create_from_file(image_path)        

    #Step 3: Detect Objects in Input Image
    detection_result = detector.detect(image)

    #Step 4: Processing and Visualizing Findings
    image_copy = np.copy(image.numpy_view())
    annotated_image = visualize(image_copy, detection_result)
    rgb_annotated_image = cv2.cvtColor(annotated_image, cv2.COLOR_BGR2RGB)

    cv2.imshow("rgb_annotated_image1", rgb_annotated_image)
      
count += 1   

#End loop after 10 seconds    
elapsed_time = time.time() - start_time  
if elapsed_time >= 6:
    break        

cap.release()
cv2.destroyAllWindows()

################################################################################ #Initialization of PoseLandmarker

⇒ This is the reason why the second loop is not executed! ! ! ! !

Without this the problem goes away

with mp_pose.Pose( static_image_mode=True, model_complexity=2, min_detection_confidence=0.5) as pose:

while True:                 
    current_time = time.time()  # 現在の時刻              
    #End loop after 10 seconds
    if current_time - start_time >= 10:
        break

cap.release() cv2.destroyAllWindows() time.sleep(1)

###################################################################################

#Execute the same object detection as the first time again

⇒ There is no problem for the first time, but an error occurs here for the second time! ! !

#Create a capture to get video from a USB camera cap = cv2.VideoCapture(0)

#Check if the camera capture was successfully opened if not cap.isOpened(): print("The camera did not open successfully!!") sys.exit()

#flame fps = cap.get(cv2.CAP_PROP_FPS) print(fps) count = 0 #frame counter start_time = time.time() #Record start time while True: ret, frame = cap.read() if not ret: #Exit the loop if the load fails break
if int(count % fps/10) == 0: #Capture and save frame images every 0.1 seconds ((fps=30)/10) path_num = int(count // fps) frame = frame.copy() #Make a copy of the frame image ######################################################
#Set the folder path as an individual save environment! ! cv2.imwrite(f"C:/Users/*Please write the folder path name!/detection_result/object_detector.jpg", frame) cv2.imshow("frame1", frame)
##################################################### #Set the folder path as an individual save environment! ! #Call the saved image to move the object detection with a still image image_path = r"C:/Users/*Please write the folder path name!/detection_result/object_detector.jpg"
img = cv2.imread(image_path)
if img is None:
print("Failed to load image!!")

    cv2.imshow("Image", img)   
    cv2.waitKey(50)
      
    #Perform object detection and visualize results
    #Step 1: Create an ObjectDetector Object
    base_options = python.BaseOptions(model_asset_path='efficientdet.tflite')
    options = vision.ObjectDetectorOptions(base_options=base_options,
                                           score_threshold=0.5)     
    detector = vision.ObjectDetector.create_from_options(options)

    #Step 2: Load Input Image
    image = mp.Image.create_from_file(image_path)        

    #Step 3: Detect Objects in Input Image
    detection_result = detector.detect(image)

    #Step 4: Processing and Visualizing Findings
    image_copy = np.copy(image.numpy_view())
    annotated_image = visualize(image_copy, detection_result)
    rgb_annotated_image = cv2.cvtColor(annotated_image, cv2.COLOR_BGR2RGB)

    cv2.imshow("rgb_annotated_image1", rgb_annotated_image)

count += 1   

#End loop after 10 seconds    
elapsed_time = time.time() - start_time  
if elapsed_time >= 6:
    break        

cap.release()
cv2.destroyAllWindows()

[kuaashish]

Hello @kinaryml,

Could you please look into this issue. Thank you

[kuaashish]

Hi @kn2234,

We have recently launched version 0.10.8 of the MP, as indicated on the release page. Kindly attempt to build in the Windows again and inform us if the issue persists on your end. If you are still seeking a resolution for this matter and it has not been resolved from your perspective, please let us know.

Thank you!

[kn2234]

Hello @kuaashish Thank you for contacting me. I tried upgrading MP version to 0.10.8. As a result, I got the same error. It doesn't seem to be improved by upgrading.

[DakeXiaoqi]

i also met this issure

[kuaashish]

Hi @joezoug,

Could you please have a look into this issue?

Thank you

[DakeXiaoqi]

    ####
    mp_image = mp.Image(image_format=mp.ImageFormat.SRGB, data=input_frame)
    detection_result = detector.detect(mp_image)
    person_results = pose_tracker.process(image=input_frame)
    pose_landmarks = person_results.pose_landmarks
    ####

in my project, I first run the object detector, and then I proceed with pose landmarks detection. Everything works fine in this sequence. However, if I reverse the order, I encounter issues. I hope this example can help you understand the problem.

[kn2234]

DakeXiaoqi Thanks for the advice. The problem I first presented is the same phenomenon. In the first routine, landmark detection is performed after object detection. After that, when the repeat routine is entered, the first step of object detection is performed after the landmark detection that was performed immediately before, so the problem occurs at that timing. The cause seems to be that some initial settings used for landmarks have been changed by performing object detection. I would like to know the solution as soon as possible.

[MalshaDeZ]

Any update on this? I'm also facing the same issue, I'm only running object detection

[bahawal32]

` # STEP 2: Create an ObjectDetector object.

    base_options = python.BaseOptions(model_asset_path='efficientdet.tflite')
    options = vision.ObjectDetectorOptions(base_options=base_options,
                                           score_threshold=0.5)`

For some reason Mediapipe is altering this path also why is it searching for the model in my environment directory when I have explicitly provided the path in base_options

RuntimeError: Unable to open file at C:\Users\bahawalkhan\Anaconda3\envs\Decorator_Pattern\lib\site-packages/E:\Decorator_Pattern\efficientdet.tflite, errno=22

@schmidt-sebastian any update on this?