Mediapipe lagging with Tkinter

[NishanthThePythoner]

I've been trying to use Mediapipe detection model with Tkinter but the result is a very laggy video. Here is the code I'm using:

import cv2
from custom_funcs import mediapipe_detection
from custom_funcs import  draw_styled_landmarks
from custom_funcs import mp_holistic 
from tkinter import *
import cv2 
from PIL import Image, ImageTk 

width, height = 800, 600

cap = cv2.VideoCapture(0)

cap.set(cv2.CAP_PROP_FRAME_WIDTH, width) 
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, height) 

app = Tk() 
app.bind('<Escape>', lambda e: app.quit()) 

label_widget = Label(app) 
label_widget.pack() 

def open_camera():
  
 with mp_holistic.Holistic(min_detection_confidence=0.5, min_tracking_confidence=0.5) as holistic:
    ret, frame = cap.read()
    image, results = mediapipe_detection(frame, holistic)
    draw_styled_landmarks(image, results)
      
  opencv_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGBA)
  captured_image = Image.fromarray(opencv_image)
  photo_image = ImageTk.PhotoImage(image=captured_image)
  label_widget.photo_image = photo_image
  label_widget.configure(image=photo_image) 

  label_widget.after(1, open_camera) 
          

button1 = Button(app, text="Open Camera", command=open_camera) 
button1.pack() 

app.mainloop()

Here's the code written in custom_funcs module:

import cv2
import numpy as np
import mediapipe as mp

mp_holistic = mp.solutions.holistic 
mp_drawing = mp.solutions.drawing_utils 

def extract_keypoints(results):
    lh = np.array([[res.x, res.y, res.z] for res in results.left_hand_landmarks.landmark]).flatten() if results.left_hand_landmarks else np.zeros(21*3)
    rh = np.array([[res.x, res.y, res.z] for res in results.right_hand_landmarks.landmark]).flatten() if results.right_hand_landmarks else np.zeros(21*3)
    return np.concatenate([lh, rh])

def mediapipe_detection(image, model):
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    image.flags.writeable = False
    results = model.process(image)
    image.flags.writeable = True
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    return image, results

def draw_styled_landmarks(image, results):
    mp_drawing.draw_landmarks(image, results.left_hand_landmarks, mp_holistic.HAND_CONNECTIONS, 
                             mp_drawing.DrawingSpec(color=(121,22,76), thickness=2, circle_radius=4), 
                             mp_drawing.DrawingSpec(color=(121,44,250), thickness=2, circle_radius=2)
                             )
    mp_drawing.draw_landmarks(image, results.right_hand_landmarks, mp_holistic.HAND_CONNECTIONS, 
                             mp_drawing.DrawingSpec(color=(245,117,66), thickness=2, circle_radius=4), 
                             mp_drawing.DrawingSpec(color=(245,66,230), thickness=2, circle_radius=2)
                             )

[NishanthThePythoner]

@kuaashish

[kuaashish]

Hi @NishanthThePythoner,

We apologize for the delay in responding. We have noticed that your current standalone code utilizes the legacy holistic solution which has been upgraded to the new Holistic Task API, which offers improved functionality. The new Task APIs simplify the process of building and customizing machine learning solutions for your applications and support for legacy Holistic solution is completely ended.

Would you be willing to try out the new HolisticLandmarker task APIs available here? Please let us know if you encounter any issues during implementation. We apologize to inform you that documentation for implementing the new Holistic Task API is not yet available, but we are actively working on it and will be out soon.

Thank you!!

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.