Enable foregrounding of 2D browser

[cabanier]

How useful would it be to show the 2D browser while in WebXR? Quest Pro added the ability to bring up the browser while the user is in an immersive game. Would it make sense to allow this while in WebXR? What are the pitfalls? (ie should it show all tabs or only the active one)

[AdaRoseCannon]

/agenda

[DR-xR]

It does make sense - you should be able to watch TV, so a web page isn't much different. However... What happens if the "2D browser" wants to show 3D content? Might there be any leak of data from one site to another?

[DR-xR]

Rik's description is that the browser window is a modal "window" - it requires dismissal before the WebXR session can accept new input.

[DR-xR]

Rik states that this is to open the web page that launched the WebXR session. Questions:

• What happens if WebXR was not directly launched from a browser?
• Must that launched browser be the default browser?
• Can the browser show 3D content?
• DOM window navigates to another site - what happens to WebXR
• What happens if the browser contains iFrame?
• User creates a new tab
  • Does it show?
  • How to go to a new page?
• Is browser shown fixed in screen-space or world-space?
• Ada's comment about controller morphing into a browser controller (mouse) requires definition of 2 WebXR controllers moving into a single (mouse-like) or touch interface. May not be an issue because the XR system already has a standard for handling controllers.

[cabanier]

• What happens if WebXR was not directly launched from a browser?

WebXR is always launched from a browser. I don't know any workflow where that is not the case.

• Must that launched browser be the default browser?

It must the browser that is running the current WebXR session

• Can the browser show 3D content?

Yes. It should have the same capabilities as the regular 2D site

• DOM window navigates to another site - what happens to WebXR

I don't believe it should act differently from a regular WebXr session. A navigation will close current session and go back to the 2D environment.

• What happens if the browser contains iFrame?

Nothing

• User creates a new tab
  • Does it show?
  • How to go to a new page?

This was mentioned during the discussion. I think the browser should show with no navigation or URL bar.

• Is browser shown fixed in screen-space or world-space?

It would be up the the UA to decide but in the case of Quest it would be world locked.

[bialpio]

My 2 cents based on the impressions from today's call: it sounds like a spin on DOM layers, so maybe we should tackle it by standardizing those?

Similarities I see:

• app uses an API to enter/configure/drive such experience
• we display a 2d plane w/ DOM that is under app's control (restricting the API to only show the requesting app's tab doesn't solve the issue of 3rd party content because iframes)

Given that it's the app that is controlling the displayed content, we probably would want to avoid entering visible-blurred state if possible (i.e. when there's no 3rd party content rendered, or maybe blur only when the user is interacting with 3rd party content?) - otherwise, it wouldn't be that much different from using the OS to multitask by bringing up a 2d browser into the immersive session.

In general, it feels a bit like we'd face the same challenges as the ones we would have to solve for DOM layers (handling input, handling 3rd party content, tackling privacy + security issues w/ no trusted pixels) that it may not make sense to make this distinct from adding a DOM layer.

[cabanier]

My intent is to bring up the 2d browser as if it's done by the OS. Maybe I'm overthinking when I proposed to limit the browser's appearance if it's triggered by JavaScript...

[alcooper91]

And would bringing up the 2D browser then allow navigation (I'd guess not as that would cause the session to unload)? Or switching tabs? It seems very much like trying to fill a role that the OS should likely fill/provide; though given that the browser is technically active for the WebXR session, it may not be selectable via those usual mechanisms?

I do think we need to tread very carefully here from a privacy/security perspective; so I don't think limiting the browser's appearance is really overthinking that much, and if we're limiting the appearance, are there things that we feel that the full browser would allow (or we could allow from a privacy/security perspective) that would be that different from what DOM Layers could allow?

[bialpio]

My intent is to bring up the 2d browser as if it's done by the OS.

My point is that if it's still the browser (as opposed to the OS) doing the actual work of displaying itself inside an immersive session, it seems to me that roughly the same work would be needed to correctly implement DOM layers and the same problems would need to be solved, so maybe we should go with that approach? :smile:

[cabanier]

And would bringing up the 2D browser then allow navigation (I'd guess not as that would cause the session to unload)?

Navigation is always allowed; even during an WebXR session. The main reason to remove the url bar and navigation UI would be to minimize user confusion but I don't feel especially strong about that.

Or switching tabs? It seems very much like trying to fill a role that the OS should likely fill/provide; though given that the browser is technically active for the WebXR session, it may not be selectable via those usual mechanisms?

Again, it would be to minimize confusion. When the OS bring us the browser through a user action, I would expect to see the full browser.

I do think we need to tread very carefully here from a privacy/security perspective; so I don't think limiting the browser's appearance is really overthinking that much, and if we're limiting the appearance, are there things that we feel that the full browser would allow (or we could allow from a privacy/security perspective) that would be that different from what DOM Layers could allow?

With DOM Layers, the XR session continues to get controller or hand input when the user interacts with the layer. Because of this, we decided that only same origin content would be allowed there. Since the session will be in visible-blurred state with no input, that concern goes away.

The other thing that is hard about layers is that each layer would be its own window with its own state which is much harder than just showing the existing window in regular state.