twitterscraper icon indicating copy to clipboard operation
twitterscraper copied to clipboard

NO RESULTS: unable to collect any tweets.

Open has-ctrl opened this issue 5 years ago • 2 comments

Just a few days ago, everything was working fine. Unfortunately since yesterday, almost all of my requests are left unanswered. This is extremely annoying because there is time pressure on my project.

This is the result for today (this is the only way it is working): INFO: queries: ['Trump since:2019-11-08 until:2019-11-09'] INFO: {'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 6.1; rv:2.2) Gecko/20110201'} INFO: Querying Trump since:2019-11-08 until:2019-11-09 INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Trump%20since%3A2019-11-08%20until%3A2019-11-09&l= INFO: Using proxy 138.197.222.35:8080 INFO: Scraping tweets from https://twitter.com/i/search/timeline?f=tweets&vertical=default&include_available_features=1&include_entities=1&reset_error_state=false&src=typd&max_position=TWEET-1192954944879550466-1192954955084304389&q=Trump%20since%3A2019-11-08%20until%3A2019-11-09&l= INFO: Using proxy 151.253.165.70:8080 INFO: Scraping tweets from https://twitter.com/i/search/timeline?f=tweets&vertical=default&include_available_features=1&include_entities=1&reset_error_state=false&src=typd&max_position=thGAVUV0VFVBaEgLup9YydjiEWisC8rcGNnY4hEjUAFQAlAFUAFQAA&q=Trump%20since%3A2019-11-08%20until%3A2019-11-09&l= INFO: Using proxy 119.192.195.83:443 INFO: Got 56 tweets for Trump%20since%3A2019-11-08%20until%3A2019-11-09. INFO: Got 56 tweets (56 new).

While this is the result for last month: INFO: queries: ['Trump since:2019-05-08 until:2019-05-09'] INFO: {'User-Agent': 'Opera/9.80 (X11; Linux i686; Ubuntu/14.10) Presto/2.12.388 Version/12.16'} INFO: Querying Trump since:2019-05-08 until:2019-05-09 INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Trump%20since%3A2019-05-08%20until%3A2019-05-09&l= INFO: Using proxy 138.197.222.35:8080 INFO: Retrying... (Attempts left: 50) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Trump%20since%3A2019-05-08%20until%3A2019-05-09&l= INFO: Using proxy 151.253.165.70:8080 INFO: Retrying... (Attempts left: 49) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Trump%20since%3A2019-05-08%20until%3A2019-05-09&l= INFO: Using proxy 119.192.195.83:443 INFO: Retrying... (Attempts left: 48) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Trump%20since%3A2019-05-08%20until%3A2019-05-09&l= INFO: Using proxy 188.226.141.127:3128 INFO: Retrying... (Attempts left: 47) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Trump%20since%3A2019-05-08%20until%3A2019-05-09&l= INFO: Using proxy 139.59.64.9:8080 INFO: Retrying... (Attempts left: 46) ...

INFO: Retrying... (Attempts left: 3) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Trump%20since%3A2019-05-08%20until%3A2019-05-09&l= INFO: Using proxy 200.89.178.186:80 INFO: Retrying... (Attempts left: 2) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Trump%20since%3A2019-05-08%20until%3A2019-05-09&l= INFO: Using proxy 207.154.231.216:3128 INFO: Retrying... (Attempts left: 1) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=Trump%20since%3A2019-05-08%20until%3A2019-05-09&l= INFO: Using proxy 200.255.122.174:8080 INFO: Got 0 tweets for Trump%20since%3A2019-05-08%20until%3A2019-05-09. INFO: Got 0 tweets (0 new).

The strangest thing is that the links actually do link to pages that have tweets (but the page returns no tweets: see below). twitter

Does anyone have any idea how to fix this?

I have already tried:

  • Reinstalling twitterscraper
  • Implementing a random delay in the retry loop

has-ctrl avatar Nov 09 '19 12:11 has-ctrl

I have the exactly same situation. Sometimes it works sometimes not, still trying to figure out why.

F1282231 avatar Nov 15 '19 08:11 F1282231

I have the exactly same situation since yesterday. I have confirmed html which was fetched and it said 'There are no results', nevertheless There are tweets if I manually searched with same query.

fimenten avatar Dec 17 '19 07:12 fimenten