ferret

ferret copied to clipboard

Describe the bug I am trying to get URLs of the all videos of the site jw.org with the script below, but Ferret throws the following exception:

Failed to execute the query operation timed out: WAIT_NAVIGATION(jw_videos) at 7:0

To Reproduce Steps to reproduce the behavior:

1. Run docker run -d -p=0.0.0.0:9222:9222 --name=chrome-headless -v /tmp/chromedata/:/data alpeware/chrome-headless-stable

2. Run ferret --cdp http://127.0.0.1:9222

3. Run the following script:

LET jw_videos = DOCUMENT("https://www.jw.org/pt/biblioteca/videos", {
    driver: "cdp",
    userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.87 Safari/537.36"
})
WAIT_ELEMENT(jw_videos, '.sectSynHdg', 20000)
CLICK(jw_videos, 'div[class*=pageSectionContainer]:nth-of-type(3) div[class*=syn-body] > h3 > a')
WAIT_NAVIGATION(jw_videos)
WAIT_ELEMENT(jw_videos, '.sectSynHdg', 10000)
FOR el IN ELEMENT(jw_videos, 'div[class*=pageSectionContainer]:nth-of-type(3) div[class*=syn-body] > h3 > a[href]')
RETURN el

Expected behavior Get URLs of the all videos of the site.

Desktop (please complete the following information):

• ferret: v0.10.1
• OS: MacOS 10.15.4
• Browser: chrome-headless-stable:latest digest: 94d864ba7845

By some reason the page crashes in headless mode and freezes in the normal one.