suddenly get 0/4/6 instead of 100 results

[telethonic]

hi,

I'm running GoogleScraper like this:

./run.py --keyword-file keywords.txt --num-results-per-page 100 --output-filename ./results.json --search-type normal --search-engines google

"keywords.txt" has a single keyword ("minecraft" in this case)

the mode used is 'http'

The number of results coming back is very variable. Sometimes it's zero (!), sometimes 4, sometimes 6, and very rarely 95-100 (I clear the cache directory before rerunning).

any idea what might be causing this?

many thanks :)

[rachs]

Hi,

I'm facing a similar issue. Im running GoogleScraper like this :

GoogleScraper -m http --keyword-file keyword_first.txt --num-workers 10 --proxy-file proxy.txt --search-engines "google,bing" --output-filename output.json -v2 -t image

'keyword_first.txt' has 7 words separated by new lines. 'proxy.txt' has 4 proxies. Despite this I get zero results for all of my keywords from both google and bing. There are places where 'num_results_for_query' in the output file is not zero, but "results" still has zero outputs. I was wondering if you were able to find a solution.

Thanks!

[telethonic]

After investigating a bit further I'm finding that the parser doesn't pull out the results correctly.

rachs, you can check you're actually getting results by looking in the files created in the .scrapecache subdirectory: unzip those files and load them in a browser.

Possibly it chokes on a result like 'Images for ...'. There could be 90+ regular results after it, but those are not returned.

[KeleiAzz]

Seems Google changed their link format. Try this: in the file parsing.py, line 451, change to 'normal': r'/url\?q=(?P<url>.*?)&sa=U&ved=',

[HugoPoi]

@KeleiAzz I try to change line 451 but no results :-(

EDIT : finally work with git version.

[pipigendut]

@KeleiAzz I try to change line 451 but no results :-(

EDIT : finally work with git version.

i still have same problem with update GoogleScraper Version