puppeteer-cluster
puppeteer-cluster copied to clipboard
When Page Crashed, throw an error, not stuck any more.
👏 Your code help me a lot,but I have some pain points with the Page Crashed Error.
When I open a page and do a lot of work,which take long long time(and I set timeout to the max value),then I would see the Page Crashed Error:
At this time,program stuck,and I can not do anything else.So I changed some code,when page crashed again,just reject~
I have a demo below,just take a try:
const { Cluster } = require('puppeteer-cluster');
(async () => {
const cluster = await Cluster.launch({
concurrency: Cluster.CONCURRENCY_CONTEXT,
maxConcurrency: 2,
timeout: 2147483647, // 32-bits max number
});
await cluster.task(async ({ page, data: url }) => {
await page.goto(url);
await page.evaluate(() => {
txt = "a";
while(1){
txt = txt += "a"; //add as much as the browser can handle
}
})
});
cluster.queue('http://www.google.com/');
await cluster.idle();
await cluster.close();
})();