puppeteer-extra icon indicating copy to clipboard operation
puppeteer-extra copied to clipboard

[Bug] goto() navigates away from requested page when stealth plugin is used

Open Audrey-Randall opened this issue 3 years ago • 1 comments

Bug description

Hello folks! I'm trying to load www.cnn.com using page.goto(). I'm using headful, stable Chrome on macOS and the stealth plugin. I expect the page to load the same way that it does on un-instrumented Chrome, pause for ten seconds, and exit. Instead, the page loads cnn.com, then appears to navigate to one of the ads that's shown further down the page, then pauses and exits. By "navigates to an ad," I mean that the page goes blank for a split second and then the ad is the only thing that reloads. It appears in the top left corner regardless of its previous position on the page. It almost looks like the page is loading the ad's iframe as the top level frame.

To reproduce, run the following snippet. If you comment out the lines that load and use the stealth plugin, the bug disappears and the page displays the expected behavior.

Code Snippet

"use strict" 

const puppeteer = require("puppeteer-extra");
// Comment out the following two lines, and the website loads as expected.
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
puppeteer.use(StealthPlugin());

const delay = ms => new Promise(res => setTimeout(res, ms));

const unexpectedNavigation = async () => {
    const executable_path = '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome';
    const browser = await puppeteer.launch({ headless: false,  executablePath: executable_path, slowMo: 250 });
    const page = await browser.newPage();
    await page.goto('https://www.cnn.com');
    await delay(10000);
    await browser.close();
}

unexpectedNavigation();

Versions

Results of npx envinfo@latest --system --binaries --npmPackages '(puppeteer|playwright*|automation-extra*|@extra*)':

System: OS: macOS 10.15.7 CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz Memory: 1.04 GB / 16.00 GB Shell: 5.7.1 - /bin/zsh Binaries: Node: 16.3.0 - /usr/local/bin/node npm: 7.15.1 - /usr/local/bin/npm npmPackages: puppeteer: ^1.6.1 => 1.20.0

Audrey-Randall avatar Jun 21 '21 22:06 Audrey-Randall

I ran into something similar, and was able to work around it by disabling the iframe.contentWindow evasion:

const StealthPlugin = require('puppeteer-extra-plugin-stealth')();
StealthPlugin.enabledEvasions.delete('iframe.contentWindow');
puppeteer.use(StealthPlugin);

hangsu avatar Oct 23 '21 01:10 hangsu

I found a similar issue loading https://k9uchicago.com/blog/page/15/

There is some kind of exit popup on this website. Before I added StealthPlugin.enabledEvasions.delete('iframe.contentWindow') the site would glitch out with the exit pop-up taking over and several console errors.

After I added StealthPlugin.enabledEvasions.delete('iframe.contentWindow'); the site renders correctly

andywaite avatar Oct 21 '22 11:10 andywaite