webxr
webxr copied to clipboard
immersive-web
Use native keyboard / Augmented Reality
I can't find a way to use the native keyboard when AR session is started.
I get that it is not useful for most VR apps, but with the new version of Chrome, AR is coming to the web with webxr, and it would be useful to get access to the keyboard.
Is it even possible?
Thank you :)
Thanks @boehm-e there may also be accessibility implications for not being able to use the keyboard alone in AR sessions. There is also a tight mapping between keyboard accessibility and alternative input devices such as switches that should be considered.
@toji I think this is interesting - but I see no follow up input from the WebXR team - so I want to ask, what do you think? How will keyboard only interactions be handled in AR by WebXR applications? Is it specified in your docs?
What do you mean by keyboard-only interactions? If you have a bluetooth keyboard connected to the device it will work as usual, the only issue is that XR content currently has no way of popping up a virtual keyboard if necessary.
There's some discussion of such an API in https://github.com/immersive-web/proposals/issues/30 , but it's worth noting: this API is wider than XR and is really useful in general.
FWIW keyboard input currently doesn't work while in immersive mode on the Oculus browser. Did we define where the keyboard input comes from?
FWIW keyboard input currently doesn't work while in immersive mode on the Oculus browser.
Oh yeah because there's nothing to type into. That's fair.
Did we define where the keyboard input comes from?
I think that's out of scope for this spec.
Anyway, I think the main thing is that because there is no accessibility tree possible for webgl content, keyboard-only interactions don't really exist for webxr. This issue is about eventually making it possible for content to request input (e.g. a "type your name" in an XR game), but I'm not sure how this relates to accessibility; this issue is about functionality that content wants.
@Manishearth you will also need to consider how users of Assistive technologies will interact, navigate etc in XR on the web - there is a mapping between keyboard only accessibility for example - which is important for may users with disabilities, as well as switch access - such as that demonstrated in this 'One thumb to rule them all' video on YT https://www.youtube.com/watch?v=2BhHwk9qSvI
NOTE: The switch demonstrated here uses a scanning application to load particular macros for the user, text or other commands.
@RealJoshue108 I'm well aware: what does this have to do with native virtual keyboards? If anything the virtual keyboards on XR devices are less accessible: there is no tactile feedback or anything, and none of them are good at being used at any decent speed since even with hand tracking it's hard to distinguish a button press from "my fingers are just laying there" when there's no physical object involved.
A separate issue is that WebGL can't have an accessibility tree in it, so keyboard only navigation has limited utility in WebXR unless content basically builds its own system. That is not what this issue is about.
FWIW keyboard input currently doesn't work while in immersive mode on the Oculus browser.
Oh yeah because there's nothing to type into. That's fair.
Yes, the bluetooth keyboard is still active but since the document is no longer in focus, no input happens.
Did we define where the keyboard input comes from?
I think that's out of scope for this spec.
Could we define an element that receives the keyboard input while in an immersive session? I agree that's likely out of scope for the first version of WebXR but we should have support later. It's feature that several people have requested.
I agree that's likely out of scope for the first version of WebXR but we should have support later. It's feature that several people have requested.
Absolutely, that's what this issue is about! I'm expressing disagreement that this issue has anything to do with accessibility.