scrapy-puppeteer icon indicating copy to clipboard operation
scrapy-puppeteer copied to clipboard

Custom puppeteer requests

Open kuwaitman opened this issue 1 year ago • 1 comments

Hi, is there a way i can custom the puppeteer request, for example to add puppeteer extra functionality?

thank you.

kuwaitman avatar Jan 30 '24 11:01 kuwaitman

Hi!

Some functionalities of Puppeteer that are not covered by our existing actions can be leveraged through CustomJsAction. This action allows you to execute custom JavaScript functions with access to Puppeteer's Page object. An example of such function can be found here.

Other scenarious require patching the service. In order to use puppeteer-extra plugins you'll have to enable them in the app.js module. Currently there are only puppeteer-extra-plugin-stealth and puppeteer-extra-plugin-recaptcha. Once enabled, you can access this functionality through CustomJsActions or by defining your own actions. For the latter, you'll need to define new PuppeteerServiceAction subclass on the client side and to add a corresponding action route in the service.

mxsnq avatar Feb 01 '24 14:02 mxsnq