Add support for OpenXR hand interaction extension

[BastiaanOlij]

This PR adds support for OpenXRs hand interaction extension that was introduced in 1.0.28.

This is a long awaited interaction that allows us to use the action system when optical hand tracking is used. This makes it much easier to create a portable application that works with hand tracking and controller tracking.

The extension introduces two new poses:

• Pinch pose is a location centered between thumb and index finger pointing forward
• Poke pose is a location at the tip of the index finger

It also adds 3 gesture based inputs:

• pinch, triggered when the user pinches thumb and index finger together. The primary use case is to use this together with a raycast childed to the pinch pose and trigger interaction with an element the raycast touches
• aim activation, is triggered when the index finger is fully extended. The primary use case is to activate a poke function when active. As the value based input is based on the curl of the index finger, the inverse can also be used to detect a trigger action.
• graps, triggered when the user makes a fist. The primary use case is to detect grabbing an object.

When both a hand interaction profile and a controller interaction profile is supplied, the XR runtime will switch between the profiles depending on whether the user is holding a controller or whether optical hand tracking is active.

However if you only supply a hand interaction profile in your action map, the OpenXR specification states that any XR runtime that supports this extension should use the hand interaction profile even if a controller is being held. This is ideal for projects that are designed toward hand tracking but want a controller fall back, especially in AR/Passthrough applications.

See: https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#XR_EXT_hand_interaction

A WIP demo project with basic functionality can be found here: https://github.com/godotengine/godot-demo-projects/pull/973

Note: This PR is basically done, I need to do some testing and create a nice example project.

[BastiaanOlij]

I've made this optional so there is now a tickbox that needs to be ticked:

I'm currently stuck with testing as I'm not sure if something is not working correctly yet, or if the extension hasn't been rolled out on Quest yet. Right now the logging isn't working properly.

[m4gr3d]

I'm currently stuck with testing as I'm not sure if something is not working correctly yet, or if the extension hasn't been rolled out on Quest yet. Right now the logging isn't working properly.

Just double-checking that you're including the hand tracking permissions when testing this extension?

[BastiaanOlij]

I'm currently stuck with testing as I'm not sure if something is not working correctly yet, or if the extension hasn't been rolled out on Quest yet. Right now the logging isn't working properly.

Just double-checking that you're including the hand tracking permissions when testing this extension?

As far as I'm aware yes, I've combined the logic with the normal hand tracking demo and the hand tracking works, but the interaction doesn't.

This could simply be because its gated behind the OpenXR version or because it's not yet enabled on the current Quest OS.

Anyway, I've added some basic logic around this here: https://github.com/godotengine/godot-demo-projects/pull/973

[BastiaanOlij]

Just a bit of extra info on the progress with this PR. Basically the functionality is finished but we're currently waiting for XR Vendors to take the implementation of the feature out of beta.

[BastiaanOlij]

Note regarding CI failure, is_hand_interaction_supported and is_eye_gaze_interaction_supported have been changed to const as these should have been const to begin with. These functions have never had mutable behaviour so not sure if we can just add those to the ignore file, or if this really breaks things?

edit I've undone this change as not to upset GDExtensions etc. It's not directly related to this PR. we can always make a separate PR if this is important.

[dsnopek]

Because the hash changes, it would really break compatibility. However, adding compatibility functions for them should be pretty easy!

There's a number of examples of them in the code base to look at, for example, TileMap::get_used_rect() had this exact same situation where it was non-const and was changed to const. Take a look at the TileMap::_get_used_rect_bind_compat_78328() method (the number is the PR), and how it's wrapped in #ifndef DISABLE_DEPRECATED and implemented in tile_map.compat.inc.

Or, if you can't work it out, just let me know and I can make a patch :-)

[BastiaanOlij]

Because the hash changes, it would really break compatibility. However, adding compatibility functions for them should be pretty easy!

There's a number of examples of them in the code base to look at, for example, TileMap::get_used_rect() had this exact same situation where it was non-const and was changed to const. Take a look at the TileMap::_get_used_rect_bind_compat_78328() method (the number is the PR), and how it's wrapped in #ifndef DISABLE_DEPRECATED and implemented in tile_map.compat.inc.

Or, if you can't work it out, just let me know and I can make a patch :-)

Was this in relation to this PR David? I think you might have meant to leave this as a comment on one of Giles PRs?

[dsnopek]

Was this in relation to this PR David? I think you might have meant to leave this as a comment on one of Giles PRs?

Yes, my comment was meant for this PR!

I was just pointing to Gilles' changes as an example for how to handle this situation, since he also had a method change to const and added the compatibility methods.

This PR will need to add some compatibility methods because adding/removing const changes the method hash, which would cause any existing GDExtension calling these methods to break.

[BastiaanOlij]

This PR will need to add some compatibility methods because adding/removing const changes the method hash, which would cause any existing GDExtension calling these methods to break.

I ended up undoing the change, it's not really related to this PR, just something I noticed for existing calls that we normally make consts. I think that if its important, we can make a separate PR.

[BastiaanOlij]

I'm changing this back to ready for review. Right now the only HMD that implements this extension is the Magic Leap 2, but I'm hoping that other headsets follow suit. I think it's worth merging this for 4.3

[akien-mga]

Thanks!