google-group-crawler
google-group-crawler copied to clipboard
Nothing happen
Hi, I tried cloned it and execute as instructed
$ pwd
/home/robbi/workplace/bash/google-group-crawler
$ export _GROUP="debmal"
$ ./crawler.sh -sh
$ ./crawler.sh -sh > curl.sh
$ /usr/bin/bash +x curl.sh
but nothing happen, inside debmal threads, mbox also empty. Maybe my steps are wrong or maybe regex on script are broken. Could you have a look on it?
Thanks for the great works!
@RobbiNespu I've just had a quick look. I believe that Google has changed [recently] their front-end application, and yeah, all the old ajax support (?_escaped_fragment_
part in URI) have just gone. When that happens, the script has no way to get any data from Google!
I will try again if there is any other work-around. You know, google promises... nothing. That's very sad :(
- Proposal (2009): https://developers.google.com/search/blog/2009/10/proposal-for-making-ajax-crawlable
- Deprecation (2015): https://developers.google.com/search/blog/2015/10/deprecating-our-ajax-crawling-scheme