secutils icon indicating copy to clipboard operation
secutils copied to clipboard

Allow users to configure trackers to bypass HTTPS errors

Open azasypkin opened this issue 1 year ago • 0 comments

Summary

Sometimes users want to track content or resources from a website that has an invalid, expired, or self-signed certificate, but Playwright, which the Web Scraper component internally relies on, fails for untrusted certificates by default.

"XXX" tracker failed to check for content changes There was an error while checking content: Failed to load page "https://xxxx.xx/": page.goto: net::ERR_CERT_DATE_INVALID at https://xxxx.xx/ Call log: - navigating to "https://xxxx.xx/", waiting until "load"

We should introduce an option to ignore such errors if the user explicitly opts into this behavior.

  const context = await browser.newContext({
    extraHTTPHeaders: options.headers,
    bypassCSP: false,
    userAgent: options.userAgent,
    ignoreHTTPSErrors: options.ignoreHTTPSErrors <---- provided by tracker
  });
  const page = await context.newPage();

azasypkin avatar Jul 06 '24 17:07 azasypkin