headless-chrome-crawler
headless-chrome-crawler copied to clipboard
How can i make customCrawl click on specific elements?
I want to make my customcrawl click on elements. They dont have a href, but a js onclick event.
Is this possible, and how and where in the code can this be done?
My code is:
const HCCrawler = require('headless-chrome-crawler');
(async () => {
const crawler = await HCCrawler.launch({
headless: true,
args: [
'--ignore-certificate-errors',
'--no-sandbox',
'--disable-setuid-sandbox',
'--disable-accelerated-2d-canvas',
'--disable-gpu'
],
maxDepth: 8,
waitUntil: "networkidle0",
customCrawl: async (page, crawl) => {
await page.setViewport({
width: 1200,
height: 800
});
const result = await crawl();
return result;
},
onSuccess: result => {
},
onError: error => {
console.log(error);
},
});
await crawler.queue(url);
await crawler.onIdle();
await crawler.close().then(() => connection.end());
})();
I think you can do this in evaluatePage function using jquery