Can't access iframe on some websites

[ysmood]

Rod Version: v0.102.0

The code to demonstrate your question

package main

import (
	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/defaults"
)

func main() {
	defaults.Show = true

	page := rod.New().MustConnect().NoDefaultDevice().MustPage("https://captcha.website")
	f := page.MustElement("div:not([style*='display:']) > iframe[data-hcaptcha-widget-id]").MustFrame()
	f.MustElement("#checkbox")
}

The code above will panic. It works fine on headless mode.

[ysmood]

A temp solution is to use PageFromTargetID to create a standalone page for the iframe:

package main

import (
	"github.com/go-rod/rod"
	"github.com/go-rod/rod/lib/defaults"
	"github.com/go-rod/rod/lib/proto"
	"github.com/go-rod/rod/lib/utils"
)

func main() {
	defaults.Show = true

	page := rod.New().MustConnect().NoDefaultDevice().MustPage("https://captcha.website")
	f := page.MustElement("div:not([style*='display:']) > iframe[data-hcaptcha-widget-id]").MustFrame()
	p := page.Browser().MustPageFromTargetID(proto.TargetTargetID(f.FrameID))
	p.MustElement("#checkbox").MustClick()
	utils.Pause()
}

[ysmood]

I think it's a bug of devtools, I don't know why Pierce: true can't get the content of the iframe:

https://github.com/go-rod/rod/blob/14ebb72947cc99e916ed97897473be2b6c64f39d/page_eval.go#L349

[KMACEL]

I encountered the same error. When I remove "show" from .rod, it works, but if the browser opens, it gets an error.

[ysmood]

I can confirm that puppeteer has the same issue, the code below will crash:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({
    headless: false
  });
  const page = await browser.newPage();
  await page.goto('https://captcha.website')
  await new Promise((r) => setTimeout(r, 5000))
  const el = await page.$("div:not([style*='display:']) > iframe[data-hcaptcha-widget-id]")
  const frame = await el.contentFrame()
  await frame.$('#checkbox').click()
})();

[tlopo]

Since this is probably a bug on CDP, does anyone know if it's being tracked somewhere ?

[ysmood]

@tlopo I have opened an issue on puppeteer, but no update yet

[jhemmmm]

Any update on this?

Your temp solution didn't work on it me.

[timestee]

The issue https://github.com/puppeteer/puppeteer/issues/8150 has been confirmed but still no update yet~

[h3isenbug]

Stuck with the same issue here. FrameID workaround did not help.

[Samler-Lee]

A temp solution is to download an older version of Chromium and replace the current version of Chromium. I successfuly tested my code with on version 884014

[alicazi]

I'm getting "Command can only be executed on top-level targets" error when PageFromTarget(proto.TargetTargetID(f.FrameID)) which f is what I got from calling el.Frame(). Wondering is this a same issue? then why the error is so different? BTW, no success with other suggested solutions (not tried old Chromium version though as I don't see it as a sustainable solution). FYI, I'm trying to create a test that needs filling credit card information in Stripe iframe form.

[ysmood]

I will take some time to redesign how the js runtime works to completely resolve this issue.

[alicazi]

@ysmood Thanks

[hsxsix]

I'm getting "Command can only be executed on top-level targets" error when PageFromTarget(proto.TargetTargetID(f.FrameID)) which f is what I got from calling el.Frame(). Wondering is this a same issue? then why the error is so different? BTW, no success with other suggested solutions (not tried old Chromium version though as I don't see it as a sustainable solution). FYI, I'm trying to create a test that needs filling credit card information in Stripe iframe form.

same problem, any updates?

[zplzpl]

当 PageFromTarget(proto.TargetTargetID(f.FrameID)) 时出现“命令只能在顶级目标上执行”错误，其中 f 是我通过调用 el.Frame() 得到的。 想知道这是同一个问题吗？那么为什么错误如此不同呢？ 顺便说一句，其他建议的解决方案没有成功（没有尝试过旧的 Chromium 版本，因为我不认为它是一个可持续的解决方案）。 仅供参考，我正在尝试创建一个需要在 Stripe iframe 表单中填写信用卡信息的测试。

same problem

[zplzpl]

A temp solution is to download an older version of Chromium and replace the current version of Chromium. I successfuly tested my code with on version 884014

worked

[shynome]

A temp solution is to download an older version of Chromium and replace the current version of Chromium. I successfuly tested my code with on version 884014

the version 884014 is not working on debian 12.

Error code: 159

---

update: the version 884014 is working on debian 11. the problem may be libc version conflict

update: the version 884014 is not work with the latest go-rod

update: Puppeteer work as expect

[ctinkong]

v0.114.5 have same problem.

[rdelcampog]

Any progress? :(

[rgunindi]

didnt work any temp solution for me.

[cplasfwst]

I have the same problem. Dear author, do you have any solution?

[ysmood]

I'm refactoring the core code of this part, I will definitely progress it within this year.

[cplasfwst]

Thank you for your hard work. I also hope to solve it as soon as possible. Is there any temporary solution that can be used now?

[ysmood]

@cplasfwst you can follow this example to use the low-level cdp api to track the tree of targets (a target is like a iframe):

https://github.com/ysmood/rod/blob/0340433eab4e6551f075eaf1875035469a777480/examples_test.go#L410-L432

You have to use the apis list in this file:

https://github.com/ysmood/rod/blob/0340433eab4e6551f075eaf1875035469a777480/lib/proto/target.go

Then you can convert a target to a rod.Page with Browser.MustPageFromTargetID.

[cplasfwst]

Thank you very much for your prompt answer. I don't quite understand the case. If my structure is like this: xframe := page.Timeout(5 * time.Second).MustElementX(//iframe[@style="border: none; display: block; visibility: visible; border-radius: 6px; overflow: hidden; position: absolute; left: 50%; top: 50%; transform: translate(-50%, -50%); width: 380px; height: 384px;"]).MustFrame() fmt.Println("检测到有验证-----------------------------------------!!!!!!!!!!!!!!!!!!qwe", xframe) xframe.MustElementX(//canvas[@id="captcha_verify_image"]).MustScreenshot(uri + /ceshi.png)

May I ask if you could provide me with a case study using my code to make it easier for me to understand? Thank you very much. I hope you can help me

[cplasfwst]

Because the code above only works normally in headless mode, memory errors will appear when displayed

[ysmood]

Sorry, I can't provide more info, it's very hard to explain. If you want to solve it you have to work on your own.

[rgunindi]

@ysmood How can we use cdp? Any kind of documentation?

[ysmood]

@rgunindi https://github.com/go-rod/rod/issues/548#issuecomment-1983067842

[hunjixin]

headless also fail， but seem not all.