aframe icon indicating copy to clipboard operation
aframe copied to clipboard

microgesture support

Open Utopiah opened this issue 7 months ago • 7 comments

The Quest Browser 38.1 support hand microgestures. They follow the OpenXR specification https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XR_META_hand_tracking_microgestures and there is a demo page at https://micro-gestures-demo.glitch.me/ and the result show up in the GamePad API https://immersive-web.github.io/webxr-samples/controller-state.html

It could be interesting, assuming that others, e.g. Apple for the VisionPro or Google for AndroidXR do follow OpenXR specs for WebXR, to bring this to a similar level than hand-tracking-controls namely extending events beyond pinchstarted, pinchended and pinchmoved with, possibly (following OpenXR naming) tap (or maybe thumb_tap), thumb_location (that I initially forgot, as I somehow didn't manage to test it) and swipe_left, swipe_right, swipe_forward, and swipe_backward new events for example.

Utopiah avatar May 09 '25 04:05 Utopiah

Related https://github.com/meta-quest/immersive-web-emulation-runtime/issues/23

Utopiah avatar May 09 '25 04:05 Utopiah

Yeah. it's on my radar. thanks

dmarcos avatar May 11 '25 05:05 dmarcos

If you didn't see it yet, @mrxz made a Connect Four game, the gestures are indeed quite nice https://x.com/noerihuisman/status/1921680416257216544

vincentfretin avatar May 15 '25 08:05 vincentfretin

Neat! I wasn't aware of it so thanks for sharing.

IMHO due to OpenXR support (and to be pragmatic, the market share of Quest...) it could arguably be into core rather than an outside component.

Utopiah avatar May 15 '25 08:05 Utopiah

Definitely a feature worth having in the core. Note that the OpenXR extension is a vendor extension by META, so there's no reason to assume other vendors will implement it. It's quite likely that other vendors would introduce their own OpenXR extension with their own gestures/naming/etc...

In our case the right abstraction is WebXR. The micro-gestures are implemented as additional buttons on the gamepad of the oculus-hand profile (see https://github.com/immersive-web/webxr-input-profiles/pull/274). So adding support is pretty straightforward.

The only question is what the right component structure would be. On one hand (pun not intended) the pattern used for controllers could be continued (meta-quest-controls uses tracked-controls, oculus-hand-tracking-controls uses hand-tracking-controls). Alternatively a component could be created specifically for microgestures, say oculus-hand-microgestures that can be added to entities with hand-tracking-controls.

mrxz avatar May 15 '25 09:05 mrxz

I would just add it to hand-tracking-controls. Maybe a property micogesturesEnabled to toggle. I assume other vendors will implement same / similar gestures. can split per vendor later if it makes sense

dmarcos avatar May 15 '25 17:05 dmarcos

Would be cool to have the Connect Four game or similar as example

dmarcos avatar May 15 '25 17:05 dmarcos