PoseOnTheGo: Approximating User Pose with Smartphone Sensor Fusion and Inverse Kinematics

Pose-on-the-Go is a full-body pose estimation system that uses sensors already found in today’s smartphones. This stands in contrast to prior systems, which require worn or external sensors. We achieve this result via extensive sensor fusion, leveraging a phone’s front and rear cameras, the user-facing depth camera, touchscreen, and IMU. Even still, we are missing data about a user’s body (e.g., angle of the elbow joint), and so we use inverse kinematics to estimate and animate probable body poses.

We have now released our app on the iOS App Store. You can download it here: https://apps.apple.com/us/app/pose-on-the-go/id1635810011

More details about the project: https://karan-ahuja.com/poseotg.html