Focus trap wars between host and iframe when both display an overlay

[Sidnioulz]

Provide a general summary of the issue here

I'm coming over from Storybook, as we've adopted RAC modals and overlays, with a peculiar bug that your own team will likely encounter on our next release :)

When a child iframe has a RAC overlay open, that traps focus, and the host document also has a RAC overlay open, both fight for control of focus and the UI is unusable by keyboard.

🤔 Expected Behavior?

This should be solved by not allowing two overlays to grab focus. Some potential solutions:

• If focus is on a parent document, RAC overlays inside a child iframe should not attempt to grab focus
• If focus is on a child iframe, RAC overlays on a host document should not attempt to grab focus
• If a child iframe loses focus, overlays in that child iframe should automatically dismiss themselves so that they don't stay open with an active focus trap

😯 Current Behavior

The overlays in the child iframe and host document fight for control of focus, resulting in flickering focus outlines and unpredictable keyboard navigation.

💁 Possible Solution

We at Storybook prefer the host popover to take over focus in these scenarios. We're okay with the child iframe overlay staying open (in fact, in our case, it's desirable), but it should not continue to grab focus.

It seems we can circumvent the issue by applying inert on the iframe as parent overlays are open, though it means we must detect when a parent overlay is open and manipulate the iframe attrs, which does not feel very elegant.

🔦 Context

No response

🖥️ Steps to Reproduce

1. Go to https://a11y-consolidation--635781f3500dd2c49e189caf.chromatic.com/?path=/story/overlay-modal--base
2. Open the modal in the preview area
3. Click on the cog icon at the top-right of the sidebar, which opens a popover
4. Press Tab

You should see focus outlines blinking between the settings popover and one of the buttons in the preview area modal.

Version

react-aria-components 1.12.1

What browsers are you seeing the problem on?

Chrome

If other, please specify.

No response

What operating system are you using?

Linux

🧢 Your Company/Team

Chromatic/Storybook

🕷 Tracking Issue

https://github.com/storybookjs/storybook/issues/32596