PoseEstimation-TFLiteSwift icon indicating copy to clipboard operation
PoseEstimation-TFLiteSwift copied to clipboard

Published 20 hours ago • verified publisher icon tucan9389
→

Metadata

The baseline project for inferencing various Pose Estimation tflite models with TFLiteSwift on iOS

logo-pose-estimatiton-for-tflite-3

platform-ios swift-version lisence PRs Welcome

This project is Pose Estimation on iOS with TensorFlow Lite.
If you are interested in iOS + Machine Learning, visit here you can see various DEMOs.

2D pose estimation in real-time 3D pose estimation

Features

  • [x] Support 2D pose estimaiton TFLite models
    • [x] Real-time demo with Metal framwork
    • [x] Photo album demo
  • [x] Support 3D pose estimation TFLite model
    • [x] Real-time demo with Metal framwork (but realtime model is not ready yet)
    • [x] Real-time pose matching demo
    • [x] Photo album demo
  • [x] Render the result keypoints of 2D pose estimation in 2D demo page
  • [x] Render the result keypoints of 3D pose estimation with SceneKit
  • [x] Render the heatmaps of 2D pose estimation output
    • [x] Part Confidence Maps for typical heatmap based models
    • [x] Part Affinity Fields for OpenPose (2D multi-person)
  • [x] Implemented pose-matching with cosine similiarity in 3D pose demo
  • [x] Implemented to fix the shoulders' keypoints in 3D pose estimation to pre-process for pose-matching

Models

Source Images

Name gif img-0 img-1 img-2
Source -

Joint Samples

Model Names gif img-0 img-1 img-2
PoseNet - pose-demo-linedot-posenet-0 pose-demo-linedot-posenet-2 pose-demo-linedot-posenet-3
PEFM CPM - pose-demo-PEFMCPM-0 pose-demo-PEFMCPM-2 pose-demo-PEFMCPM-3
PEFM Hourglass - pose-demo-PEFMHourglass-0 pose-demo-PEFMHourglass-2 pose-demo-PEFMHourglass-3
OpenPose (multi-person) - pose-demo-openpose-0 pose-demo-openpose-2 pose-demo-openpose-3

Heatmap-ConfidenceMap Samples

Model Names gif img-0 img-1 img-2
PoseNet - - - -
PEFM CPM - pose-demo-heatmap-PEFMCPM-0 pose-demo-heatmap-PEFMCPM-2 pose-demo-heatmap-PEFMCPM-3
PEFM Hourglass - pose-demo-heatmap-PEFMHourglass-0 pose-demo-heatmap-PEFMHourglass-2 pose-demo-heatmap-PEFMHourglass-3
OpenPose (multi-person) - pose-demo-heatmap-posenet-0 pose-demo-heatmap-posenet-2 pose-demo-heatmap-posenet-3

Heatmap-PAF Samples

Model Names gif img-0 img-1 img-2
OpenPose (PAF x) - pose-demo-heatmap-pafx-PEFMHourglass-0 pose-demo-heatmap-pafx-PEFMHourglass-2 pose-demo-heatmap-pafx-PEFMHourglass-3
OpenPose (PAF y) - pose-demo-heatmap-pafy-PEFMHourglass-0 pose-demo-heatmap-pafy-PEFMHourglass-2 pose-demo-heatmap-pafy-PEFMHourglass-3

Meta Data

2D

✅ vs ☑️ Name Size Inference
Time
on iPhone11Pro		 Post-process
Time
on iPhone11Pro		 PCKh-0.5 multi person
vs
single person		 Model Source Paper tflite
download
✅ PoseNet 13.3 MB - - - single tensorflow/tensorflow PersonLab
✅ PEFM CPM 2.4 MB - - - single edvardHua/PoseEstimationForMobile Convolutional Pose Machines
✅ PEFM Hourglass v1 1.8 MB - - - single edvardHua/PoseEstimationForMobile Stacked Hourglass Networks
✅ PEFM Hourglass v2 1.7 MB - - - single edvardHua/PoseEstimationForMobile Stacked Hourglass Networks
✅ OpenPose 7.8 MB - - - multi ildoonet/tf-pose-estimation OpenPose
☑️ AlphaPose - - - - single osmr/imgclsmob RMPE
☑️ SelecSLS - - - - single osmr/imgclsmob -
☑️ IBPPose - - - - single osmr/imgclsmob -
☑️ Lightweight OpenPose - - - - single osmr/imgclsmob OpenPose

3D

✅ vs ☑️ Name Size Inference
Time
on iPhone11Pro		 Post-process
Time
on iPhone11Pro		 (metric) Model Source Paper tflite
download
✅ Baseline3DPose 137.1 MB 347 ms 79 ms - mks0601/3DMPPE_POSENET_RELEASE Baseline3D download
✅ LiteBaseline3DPose 16.6 MB 116 ms
(cpu only)		 19 ms
(cpu only)		 SangbumChoi/MobileHumanPose MHP download

Requirements

  • Xcode 11.3+
  • iOS 11.0+
  • Swift 5
  • CocoaPods
gem install cocoapods

Build & Run

  1. Install dependencies with cocoapods
cd ~/{PROJECT_PATH}
pod install
  1. Open the PoseEstimation-TFLiteSwift.xcworkspace and run the project

See also

License

This repository is licensed under Apache 2.0. Full license text is available in LICENSE.

← Metadata

146
Stars
20
Forks
Watchers

Owner

verified publisher icon tucan9389

Metadata

The baseline project for inferencing various Pose Estimation tflite models with TFLiteSwift on iOS

Back