HippoYD
HippoYD copied to clipboard
Published
20 hours ago •
iglaweb
iglaweb
Mouth openness classifier trained with TensorFlow and video dataset YawDD
HippoYD: Yawn Detector using mouth
Overview
This is a mouth openness detection model. Full training code, data preparation scripts, and pretrained models are in the repository.
Open image inference in Colab.
Demo
Requirements
- Refer to requirements.txt for dependent libraries that're needed to use the code.
Image Dataset
The model was trained using two sources:
- Main source: Video dataset YawDD
Paper: S. Abtahi, M. Omidyeganeh, S. Shirmohammadi, and B. Hariri, “YawDD: A Yawning Detection Dataset”, Proc. ACM Multimedia Systems, Singapore, March 19 -21 2014. - Augmentered by Kaggle Drowsiness_dataset.
How to train
- Convert YawDD dataset to image folders, 2 classes:
closedandopened
python convert_dataset_video_to_mouth_img.py
- Split data into 3 datasets:
train,validation,test
python split_data_into_datasets.py
- Train data with:
python train_yawn.py
Model evaluation
| Dataset | AC | PR | RC | F1 |
|---|---|---|---|---|
| Kaggle | 0.932 | 0.992 | 0.871 | 0.927 |
| Test split (5%) | 0.988 | 0.960 | 0.975 | 0.967 |
Available pretrained models [full, lite] and demos
| Model | Example of inference | Demo |
|---|---|---|
| SavedModel / Keras H5 | run_yawn_inference_tf_h5.py | |
| TFLite | run_yawn_inference_tflite.py | |
| TensorFlowJS | image_predict.js | https://igla.su/mouth-open-js/ |
| ONNX | run_yawn_inference_onnx_cv.py run_yawn_inference_onnx_onnxruntime.py |
https://igla.su/mouth-open-js/ |
| Frozen pb | run_yawn_inference_tf_pb.py |
Model Inference Results
| Configuration | Config | Model | Time (avg) | TFLite ver. |
|---|---|---|---|---|
| Macbook Pro, CPU 2 GHz Quad-Core Intel Core i5 |
CPU | TFLite (Floating) | 5 ms | 2.3 |
| CPU | TFLite (Quantized) | 8 ms | 2.3 | |
| CPU | Keras H5 (Floating) | 30 ms | 2.3 | |
| CPU | ONNX | 2 ms | 2.3 | |
| CPU | Frozen pb | 4 ms | 2.3 | |
| Wasm (Safari 14.0, Firefox 84) | TensorFlowJS | 30 ms | 2.3 | |
| Xiaomi MI8 | GPU/CPU 3 Threads | TFLite (Floating) | 4 ms | 2.4 |
| CPU 3 Threads | TFLite (Quantized) | 10 ms | 2.4 | |
| Xiaomi Redmi 9 | GPU/CPU 3 Threads | TFLite (Floating) | 11 ms | 2.4 |
| CPU 3 Threads | TFLite (Quantized) | 9 ms | 2.4 | |
| Samsung Galaxy S20+ | GPU/CPU 3 Threads | TFLite (Floating) | 2 ms | 2.4 |
| CPU 3 Threads | TFLite (Quantized) | 2 ms | 2.4 |
Issues
If you find any problems or would like to suggest a feature, please feel free to file an issue
License
Copyright 2021 Igor Lashkov
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
iglaweb