SerpScraper
SerpScraper copied to clipboard
:mag_right: Google/Bing search results scraping using PHP. Tested and working / May 13, 2020
This is the quote from here: https://github.com/Athlon1600/SerpScraper/blob/78c0f671f3e2cb213616bc4e6f28dbc80223a021/src/Browser.php#L17 I assume it bans IPv6 faster? What if you have a subnet of /48?
It seems like a new Google Cookie popup is stopping this working, see screenshot.  That's what you get when you echo the response.
More of a feature request, but might be worth having an addition to the array to say if the output URL is a advert or not. Also would be good...
Hello, I'm also finding that Google sending 200 HTTP CODE but is not the real one, see this results ``` [1] => Array ( [url] => https://www.google.co.id/search?gl=id&q=domain+murah&oq=domain+murah&start=0&client=psy-ab&gbv=1&complete=0&num=100&pws=0&nfpr=1&ie=utf-8&oe=utf-8 [referer] => https://www.google.com/sorry/index?continue=https://www.google.co.id/search%3Fgl%3Did%26q%3Ddomain%2Bmurah%26oq%3Ddomain%2Bmurah%26start%3D0%26client%3Dpsy-ab%26gbv%3D1%26complete%3D0%26num%3D100%26pws%3D0%26nfpr%3D1%26ie%3Dutf-8%26oe%3Dutf-8&q=EgRrmMAfGLGDj_oFIhkA8aeDS5MOwTuCq_WmvtVEq7jIxMQJPHNLMgFy...
Hello, Sometime for certain proxy google will sending back mobile version and the code on GoogleSearch.php is only trying to find desktop version I had use this pattern to get...
Fatal error: Uncaught Error: Class 'Curl\BrowserClient' not found in D:\htdocs\SerpScraper\Browser.php:7 Stack trace:
Hi! Everything is working except this : $bing->setPreference('results_per_page', 50); It gives me only 10 results with 'en-us' but it works with 'en-ca' or 'fr-fr' and other where I have 50...
Not an issue, but a feature request to possibly add the ability to remove google ads results from the top of the results i.e.: ``` [0] => https://www.google.com/aclk?sa=l&ai=DChcSEwitwarhuMjpAhVpwRYFHQepBhwYABAEGgJ0bA&sig=AOD64_3a6AMkN20suIOK-MrMx-E1m-_DRA&ved=2ahUKEwitnqLhuMjpAhUs7XMBHTMBA4gQpigoAHoECGgQBA&adurl= [1] =>...
Hi, does the script can specify a region? most of the time it auto detects base on IP but would it be possible to specify what region it is?
I suppose there is a typo here ? ``` function setProxy($proxy, $new_profile_id = true){ if($proxy == false){ $this->options[CURLOPT_PROXY] = null; } else { $this->options[CURLOPT_PROXY] = $proxy; } // do we...