virtual-puppet-project
[FEATURE]: Mediapipe support
Description
Add mediapipe as a supported tracker. Mediapipe holistic + iris tracking provides full face and body tracking.
Why is this needed
Mediapipe only requires a webcam to track a person's face and body.
Additional information
GDMP - A gdnative integration for mediapipe that works on both windows and linux
Has there been any progress on this? It seems like it'd be a pretty significant upgrade :thinking:
I have been able to build the module into the project, and i made a basic extension GUI that interrogates just like all the other extensions but i get lost when trying to connect the tracking data from media pipe into the rest of the program. I also never figured out the best way to translate the media pipe tracking data, there are a lot of examples online on how the math works but that would also need to be done
The latest commit on master adds preliminary mediapipe support (that's still broken on vpuppr's side) via GDMP.
The relevant files/lines are:
- libvpuppr @ vrm_puppet.rs for applying data to a model
- vpuppr @ trackers/media_pipe.gd for starting MediaPipe
- vpuppr @ gui/tracking/tracking.gd for connecting tracking data to the model
- vpuppr @ addons/GDMP for the actual Godot addon