aframe
aframe copied to clipboard
aframevr
Hand tracking choppy on Vision Pro
It feels jittery / low framerate. Any other headset is fine (e.g Quest). Can test on https://aframe.io/aframe/examples/showcase/hand-tracking/ and compare Quest3 and Vision Pro. It's difficult to capture on video but Quest 3 hand tracking much smoother
Also hand model is distorted. Might be something particular to Vision Pro joint tracking we're not accounting for
Thoughts about distortion. I vaguely remember saying @rcabanier mentioning that Quest doesn't return the joints data that exactly match user's hands for privacy reasons. Instead a generic hand size is returned. Wonder if the discrepancies I'm seeing are related to the different approaches between Meta Browser and Safari. It's subtle and hard to capture in video.
Did a second test and Quest 3 hand tracking definitely feels less jittery than Vision Pro with the example linked above: https://aframe.io/aframe/examples/showcase/hand-tracking/
This might not be a bug in Safari or A-Frame just differences between hand tracking systems. I captured a video in each headset to compare. Ignore the color differences (separate issue) Videos were taking in same room, in same lighting conditions just a few mins apart.
Vision Pro
https://github.com/aframevr/aframe/assets/39342/b9bc64f8-5a3c-4963-80f6-f4535bb54375
Quest 3
https://github.com/aframevr/aframe/assets/39342/6b5a0560-eec3-4fdd-8911-47cacbb31b63
Read somewhere that hand tracking only runs at 30Hz on Vision Pro. Might be part of the issue.
I also made a comparison video: https://www.youtube.com/watch?v=bcdmz-gW7HM (real and virtual synced manually, so don't judge it by lag).
Vision Pro is more accurate when i join hands.
Anyway, the issue on this topic is a very known problem of VisionPro: it update hand-tracking only at 30hz.
looks like hand tracking will get 60Hz updates in Vision OS 2. Hopefully this improves
Hand tracking on visionOS 2 should be updated at display refresh rate / 90Hz.
visionOS 2 video: https://github.com/aframevr/aframe/assets/293795/d8f9e202-c06f-42e1-9672-5ad91d9c44bf
