Timeout error, wait_for_selector

[Armandd1]

Hello, until now I used the syncron version and I'm now trying to switch to asyncron, but I keep experiencing the following problem, can anyone help me?

Example program: async def simple_crawl(): async with AsyncWebCrawler() as crawler: result = await crawler.arun( url="https://visitmures.com/en/events", bypass_cache=True, page_timeout=90000, ) print(result.markdown)

asyncio.run(simple_crawl())

Console: [ERROR] 🚫 arun(): Failed to crawl https://visitmures.com/en/events, error: [ERROR] 🚫 crawl(): Failed to crawl https://visitmures.com/en/events: Page.wait_for_selector: Timeout 30000ms exceeded. Call log: waiting for locator("body") to be visible

• locator resolved to hidden …
• locator resolved to hidden …
• locator resolved to hidden …
• locator resolved to hidden …
• locator resolved to hidden …

[unclecode]

Hi @Armandd1 Thank you for using our library and for pointing this out. After investigating, I found the cause of the issue: the website you're trying to crawl uses some tricks, including keeping the body element invisible. We wait for this element to be visible, which caused the problem. This has already been fixed, and I've added a new 'ignore_body_visibility' flag that can be set to true if the body's visibility doesn't change. This allows us to manually make the body visible. I've tested this and it works for your website. If needed, this flag can also be used to override the default behavior. No code changes are required from you; just wait for the updated 0.3.73 version, which will be released in a couple of days. Thanks again.

[aravindkarnam]

Closing this issue due to inactivity. Looks like this issue is now resolved in newer versions.