ofxFaceTracker3
ofxFaceTracker3 copied to clipboard
Fast and robust face tracking addon for openFrameworks based on YOLO5Face
ofxFaceTracker3
Working in progress
Fast and robust face tracking addon for openFrameworks based on YOLO5Face and ONNX Runtime.
Features
- Fast and robust face & keypoints detection using YOLO5Face.
- Achieve realtime FPS on both CPU and GPU.
Tested environment
- oF0.11.2 + macOS Catalina Intel CPU
- oF0.11.2 + Windows10 CPU / CUDA / TensorRT
- CUDA 11.4, TensorRT 8.0.3.4
Installation
- This addon depends on following addons. Please pull them to
${OF_BASE_PATH}/addons
directory first. - Generate project using project generator, then
model
directory is copied intobin/data
.
Usage
- In
model
directory, there are 2 converted pretrained models, which areyolov5s-face_640x640.onnx
andyolov5n-face0.5_320x320.onnx
.-
yolov5s-face_640x640.onnx
is suitable for GPU detection, andyolov5n-face0.5_320x320.onnx
is suitable for CPU detection with slightly accuracy degradation. - Original PyTorch pretrained models can be downloaded from here.
- Then onnx files are generated using this script.
-
-
ofxFaceTracker3::Tracker::setupCpu();
is handy setup method for CPU detection, which loadsyolov5n-face0.5_320x320.onnx
by default. -
ofxFaceTracker3::Tracker::setupGpu();
is handy setup method for GPU detection, which loadsyolov5s-face_640x640.onnx
by default. - See
example-facetracker3
for more details.
Notes
- If TensorRT is enabled, it takes long time when starting app for the first time. In my environment, it takes 12 minutes. Then converted
*.trt
file is generated underbin/data/model/yolov5s-face_640x640_trt_cache
directory.
Comparison to ofxFaceTracker2
- For easy environment such as no difficult lighting, no occulusion and frontal angle, ofxFaceTracker2 might be better because it runs on CPU with super lightweight load.
- However ofxFaceTracker2 hardly detects masked faces which are common in COVID-19 era, and also it does not support non-frontal faces.
- If you guys face any of difficult detection conditions, ofxFaceTracker3 will perform lots better than ofxFaceTracker2.
Reference
- I heavily referred Lite.AI.ToolKit implementation.