Thumbs up animations

[achleey]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

macOS 14.1.2 (Apple M1 Pro)

MediaPipe Tasks SDK version

No response

Task name (e.g. Image classification, Gesture recognition etc.)

Hand Landmark and Gesture Recognition

Programming Language and version (e.g. C++, Python, Java)

Python

Describe the actual behavior

When trying to classify a thumbs up as a letter A (in Guatemalas Sign Language) the thumbs up animation from the demo comes up. When showing the 2 thumbs up, a firework animation comes up. When showing the peace sign (letter V in Guatemalas sign language) the victory animation comes up

Describe the expected behaviour

I wish there was no animation shown so I can code my project correctly. How can I modify it so it won't show?

Standalone code/steps you may have used to try to get what you need

import pickle

import cv2
import mediapipe as mp
import numpy as np

model_dict = pickle.load(open('./model.p', 'rb'))
model = model_dict['model']

cap = cv2.VideoCapture(1)

mp_hands = mp.solutions.hands
mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles


labels_dict = {0: 'A', 1: 'B', 2: 'C'}

while True:

    ret, frame = cap.read()

    H, W, _ = frame.shape

    frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

    with mp_hands.Hands(static_image_mode=True, min_detection_confidence=0.5) as hands:
        results = hands.process(frame_rgb)

        if results.multi_hand_landmarks:
            for hand_landmarks in results.multi_hand_landmarks:
                mp_drawing.draw_landmarks(
                    frame,
                    hand_landmarks,
                    mp_hands.HAND_CONNECTIONS,
                    mp_drawing_styles.get_default_hand_landmarks_style(),
                    mp_drawing_styles.get_default_hand_connections_style()
                )

                data_aux = []
                for landmark in hand_landmarks.landmark:
                    x = landmark.x
                    y = landmark.y
                    data_aux.append(x)
                    data_aux.append(y)

                prediction = model.predict([np.asarray(data_aux)])
                predicted_character = labels_dict[int(prediction[0])]

                x1 = int(min(np.asarray(data_aux)[::2]) * W)
                y1 = int(min(np.asarray(data_aux)[1::2]) * H)
                x2 = int(max(np.asarray(data_aux)[::2]) * W)
                y2 = int(max(np.asarray(data_aux)[1::2]) * H)

                cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 0, 0), 2)
                cv2.putText(frame, predicted_character, (x1, y1), cv2.FONT_HERSHEY_SIMPLEX, 1.3,
                            (0, 0, 0), 2, cv2.LINE_AA)

    cv2.imshow('Frame', frame)

    if cv2.waitKey(25) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Other info / Complete Logs

The animation shows approximately 5 seconds of the hand being on frame.

[kuaashish]

Hi @achleey,

After reviewing the code snippet you provided, it appears that you are using an legacy version of MediaPipe Hand Solution. We have recently launched enhanced MediaPipe Tasks APIs, which provide increased stability and functionality compared to the legacy version.

We strongly advise migrating to these updated APIs for Hand landmarks, as we no longer support legacy solutions. You can find the documentation for the Hand Landmarker here and implementation guide for Python here.

Unfortunately, aside from this, we won't be able to assist you much further.

Thank you!!

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[konecnyna]

If you're on macos you need to disable reactions in video settings unrelated to mediapipe.

[achleey]

@konecnyna thanks so much! That helped me resolve the issue! I can't believe that was the problem