Nested `Dialog` inside `AnchoredOverlay` dismisses both

[adierkens]

Description

When a Dialog is launched from an AnchoredOverlay interacting w/ the Dialog causes both to be dismissed.

This might be due to the Dialog not leveraging the Overlay component to manage state (which had tests/stories for nested dialogs). I'm able to work around this using the ignoreClickRefs options for overlayProps, but should work OOTB

https://github.com/user-attachments/assets/a2da3eb5-014f-44c6-ada6-134f8e50a000

Steps to reproduce

https://stackblitz.com/edit/react-rwpvek2f?file=Demo.tsx

Create an AnchoredOverlay that launches a Dialog w/ interactive elements.

    <AnchoredOverlay
      open={open}
      onOpen={() => setOpen(true)}
      onClose={() => setOpen(false)}
      renderAnchor={(props) => <Button {...props}>Toggle overlay</Button>}
    >
      <div>
        Anchored overlay content
        <Dialog>
          Test 123
          <FormControl>
            <FormControl.Label>Label</FormControl.Label>
            <TextInput />
          </FormControl>
        </Dialog>
      </div>
    </AnchoredOverlay>

Version

v37.30

Browser

No response