support iframes + better debugging dev ex in dom processing

[navidkpr]

why

We want to be able to extract elements from iframes and use them for extraction / taking action.

This PR unlocks iframe support overall, but iframe access is still blocked on browser-level (by iframe same origin policy) for most websites. We will need to override this in our own browser to unlock the full power of this PR.

what changed

When processing doms, we also process elements from iframes. For visibility checks, we check if the parent iframe is visible and if the element is visible inside the iframe. For element in view, we now calculate the element height based on the iframe window location.

test plan

Tested locally by running on websites with extensions that added iframes. You can test this by installing https://system32.sh on your browser and checking if the dom processor extract elements from the extension on top of the page.

[changeset-bot[bot]]

🦋 Changeset detected

Latest commit: 50f8408f88b155a30d178c25ae40145b4c10ba20

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@browserbasehq/stagehand	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[neoromantic]

Just curious, would it support nested iframes? I have such case in my work with stagehand/browserbase

[gorlitzer]

Hello there, any update on this?

[Szetty]

@navidkpr @pkiv @kamath @seanmcguire12 any updates ?

[Szetty]

@seanmcguire12 any reason why this was closed ? maybe it was resolved differently ?

[seanmcguire12]

@Szetty this was closed because it wasn't the approach we wanted to take for solving the iframe problem. As noted in the PR desc, it'll still be blocked by the same origin policy, so it didn't make much sense to merge before having this part solved.

We're currently working on a different approach that uses CDP instead of injected/browser side JavaScript so that we can get around this.

The current workaround for this is to use stagehand.agent with a computer-use model. Hoping we'll be able to deliver a solution that works with the classic stagehand primitives (act, extract, observe) very soon though, so stay tuned!