se-scraper icon indicating copy to clipboard operation
se-scraper copied to clipboard

Set different geolocation for every request

Open mpassion opened this issue 5 years ago • 2 comments

I need to set different gelocation for every search query to Google. I set manually permissions in Browser.js (puppeteer-cluster catalog) because in se-scraper I don’t have access to browser object (or maybe I’m wrong?)

context = yield chrome.createIncognitoBrowserContext(); 
context.clearPermissionOverrides();  
context.overridePermissions('https://www.google.pl', ['geolocation']);

Next I overwritten GoogleScraper due to extends search_keyword method with geolocation params:

async search_keyword(keyword) {
	await this.page.setGeolocation({
	           latitude: latVar,
	           longitude: longVar
	});
	const input = await this.page.$('input[name="q"]');
	await this.set_input_value(`input[name="q"]`, keyword);
	await this.sleep(50);
	await input.focus();
	await this.page.keyboard.press("Enter");
}

I noticed that it’s necessary to reload the page to update Google results with new geo. When I put this code after Enter in search_keyword

await this.page.evaluate(() => {
	location.reload(true)
});

I got: Error: Execution context was destroyed, most likely because of a navigation

I have also tried with put

await input.focus();
await this.page.keyboard.press("Enter");

but the same as above.

How to properly refresh the page or how to get the results related with geo params?

mpassion avatar Jul 11 '19 13:07 mpassion

Same issue as above.

Krajstofer avatar Jul 15 '19 08:07 Krajstofer

Honestly I don't know what the solution is for this. I don't have currently the time to debug this, can anyone step in?

NikolaiT avatar Jul 18 '19 16:07 NikolaiT