how to use find_xpath inside an iframe

[BlinkyStitt]

Hello! Thanks for this useful crate!

I'm trying to automate clicking around inside of an iframe. Searching around online has given me a bunch of different answers and none have worked, so I figured it was time to open an issue.

I have the browser launched and the page loaded, then I use chromiumoxide to click some buttons and get the iframe opened. But I'm not sure how to automate the rest.

This selector finds the iframe. I can take a screenshot of it and it displays what I expect.

let iframe_element = page.find_xpath(format!("//iframe[@src='{}']", my_frame_url().trim_end_matches("/"))).await?;

But I can't search xpaths inside of the iframe. I am sure that this is because I need to change the ExecutionContextId to be the frame's instead of the main page's. But how?

With head enabled, I can manually click around in the browser window. In the developer tools, I click "Top" in the Execution Context Selector and then choose the iframe's context. Then I can use the console to click() and other things. Since I can do this by using the developer tools in the GUI, I assume I should be able to use chromiumoxide to do the same. Hopefully I'm just missing a function call and don't need to patch chromiumoxide.

I was hoping that something similar to this would work:

# launch the browser, open the page, click some buttons, wait for the iframe to load...

let targets = browser.fetch_targets().await?;

let iframe_target = targets
    .into_iter()
    .find(|target| target.url.trim_end_matches("/") == my_frame_url().trim_end_matches("/"))
    .ok_or_else(|| eyre::eyre!("iframe target not found"))?;

let iframe_page = browser.get_page(iframe_target.target_id).await?;

// my real app will use a more specific selector, but this is the general idea
iframe_page.find_xpath('//button').await?.click().await?;

Sinct the iframe target isn't a "Page", the let iframe_page line fails with an error. Also, the TargetInfo returned by browser.fetch_targets has a BrowserContextId, but not an ExecutionContextId.

What am I missing? Or does something need to be added to chromiumoxide to make this possible? Or is there a workaround?

The target's type is Unknown(iframe), so that makes me think there will be at least a small patch needed.

I also noticed that after I run browser.fetch_targets(), I can't use the page anymore. It says the receiver is closed.

Thanks!

---

If I turn off some security settings, the iframe shows up in the frametree instead of in the targets. That's different, but I don't think it helps me. Since I can do what I need by manually clicking around in the GUI, I don't think this is the right path for me.

let browser_config = BrowserConfig::builder()
    .arg("--disable-site-isolation-trials") // Disable Site Isolation
    .arg("--disable-features=IsolateOrigins,site-per-process") // Prevents iframe process separation
    ...

let frame_tree = page.execute(GetFrameTreeParams {}).await?.result.frame_tree;
println!("Frame Tree Child Frames: {:#?}", frame_tree.child_frames);

[BlinkyStitt]

with the thirtyfour crate, i use an xpath to get the iframe element, then I call iframe_element.enter_frame().await

That sends a command:

            Command::SwitchToFrameElement(element_id) => {
                RequestData::new(Method::POST, format!("session/{}/frame", session_id)).add_body(
                    json!({"id": {
                        "ELEMENT": element_id.to_string(),
                        MAGIC_ELEMENTID: element_id.to_string()
                    }}),
                )
            }

I'm not sure where chromiumoxide's equivalent request posting is.

[BlinkyStitt]

There's also 3 types of ids on our Element. I'm not sure which is supposed to be sent.

#[derive(Debug)]
pub struct Element {
    /// The Unique object identifier
    pub remote_object_id: RemoteObjectId,
    /// Identifier of the backend node.
    pub backend_node_id: BackendNodeId,
    /// The identifier of the node this element represents.
    pub node_id: NodeId,
    tab: Arc<PageInner>,
}

[PleaseDont]

Have you found a solution?

[Sytten]

@BlinkyStitt If you found a solution, feel free to share. Otherwise I will close as nothing is actionable for now.