Hand Tracking Feature Discussion

[TiborUdvari]

Nature of issue?

• [ ] Found a bug
• [ ] Existing feature enhancement
• [x] New feature request

Which area does this problem relate to?

• [x] VR
• [x] AR(ARCORE)
• [x] AR(MARKER)

New feature details:

I'm looking into implementing hand tracking. According to the specification, WebXR provides the following skeleton joints of each hand.

Here is a handy example implementation: demo source

## Proposed v1 p5.xr Behaviour

It should mimic the p5 touches API, to keeps a global updated list. The difference here is that we have orientation information as well, which might be useful.

Most interaction would focus on the finger tips, which should be easily accessible, and indexed according to a Five Finger System as in keyboard instruments, to increase access.

Hand tracking would be passed in as an optionalFeatures and nothing would happen if it is not active.

I would start with implementing an API like this

leftHand // array containing left hand skeletal joint positions
rightHand // array containing left hand skeletal joint positions
leftFingers // array containing left finger tip positions indexed my Five Finger System
rightFingers // array containing left finger tip positions indexed my Five Finger System
leftIndexFinger // position of left index finger
rightIndexFinger // position of right index finger