twitterscraper 0 tweets

trafficstars

INFO: Retrying... (Attempts left: 1) INFO: Scraping tweets from https://twitter.com/search?f=tweets&vertical=default&q=bitcoin&l= INFO: Using proxy 181.211.38.62:47911 INFO: Got 0 tweets for bitcoin.

Parsing may be an issue. Both twitterscraper (0.9.3) and (1.4.0) are failing.

Jun 02 '20 04:06 etemiz

Have also been facing this issue. Queries that were returning tweets yesterday are not returning tweets today.

Jun 02 '20 05:06 mickyscreggs

I'm also facing the Same issue! Yesterday it was parsing well, but today it returns 0 tweets

Jun 02 '20 06:06 ravishankarramakrishnan

same here 0 tweets

Jun 02 '20 06:06 xtr32

same here 0 tweets

Jun 02 '20 06:06 yiw0104

Seems Twitter has restricted the connection so that all requests return a page with "We've detected that JavaScript is disabled in your browser. Would you like to proceed to legacy Twitter?"

Jun 02 '20 06:06 tengfei7890

+1. That's bad.

Jun 02 '20 09:06 panoptikum

hocam bende bir proje geliştirmiştim projemde ana kısım buna bağlı bu sorunu nasıl düzeltebiliriz

Jun 02 '20 12:06 hakanyusufoglu

I need help

Jun 02 '20 12:06 hakanyusufoglu

same here... anyone has a clue for whats going on?

Jun 02 '20 13:06 toscanopedro

Not yet. I used it for school university project. What will I do during the presentation

Jun 02 '20 13:06 hakanyusufoglu

Indeed, this can be fixed by modifying the header dictionary in query.py from HEADER = {'User-Agent': random.choice(HEADERS_LIST)} to HEADER = {'User-Agent': random.choice(HEADERS_LIST), 'X-Requested-With': 'XMLHttpRequest'} that should fix the issue.

Jun 02 '20 15:06 rubengoeminne

It works for me! Thanks @rubengoeminne, genius!

Jun 02 '20 16:06 locchipinti

Thanks

I am very thank you. its work.

Jun 02 '20 16:06 hakanyusufoglu

its work.. thanks

Jun 02 '20 16:06 xtr32

hi guys, im a kind of noob and do not have a HEADER in my code... someone can tell how can i implement it?

Jun 02 '20 16:06 toscanopedro

Thanks a lot my friend! This worked for me! You are a genius! Let me share you a beer @rubengoeminne! Paulaner German Beer? Or Negra Modelo Mexican Beer?

Jun 02 '20 19:06 GivenToFlyCoder

@toscanopedro The header dictionary: HEADER = {'User-Agent': random.choice(HEADERS_LIST)} is not in your own code, instead it is a line inside the file query.py

Just open the file as TXT, and change the lines, such as @rubengoeminne said. You could search the file in your PC, maybe it will be foun at the path: C:\ProgramData\Anaconda3\Lib\site-packages\twitterscraper

Jun 02 '20 19:06 GivenToFlyCoder

THX MAN!!!!

Jun 02 '20 22:06 toscanopedro

The modification no longer works for query_user_info. I changed the header dictionary in query.py and still got no information on my list of users.

Jun 02 '20 23:06 yiw0104

I faced the same issue. It seems to work now to retrieve the tweets. However I get this error when I want to have user info, using query_user_info : local variable 'user_info' referenced before assignment

Jun 03 '20 01:06 AlexBietrix

Yah it is not working for me. Changed that line in query.py and same issue occurs.

Jun 03 '20 01:06 mardiaz353

Hi. I have implemented the modification suggested by pumpkinw and the algortihm made progress. It was not scraping anything before modification. But after modification it is scraping, but not everything. It seems it is scraping only some last hours. For example, when I issued:

twitterscraper fascismo --lang pt -p 1 -bd 2020-05-31 -ed 2020-06-01 -o file_name.json

I received tweets corresponding only to hours from 20 up to 23 of day 2020-05-31:

In [12]: df.groupby(df['timestamp'].dt.hour).count()

Out[12]: has_media hashtags img_urls is_replied ... tweet_url user_id username video_url timestamp ...
20 956 956 956 956 ... 956 956 956 956 21 2384 2384 2384 2384 ... 2384 2384 2384 2384 22 2100 2100 2100 2100 ... 2100 2100 2100 2100 23 2147 2147 2147 2147 ... 2147 2147 2147 2147

[4 rows x 21 columns]

Somebody know what is going on?

Jun 03 '20 03:06 wal-iston

already changed the header from HEADER = {'User-Agent': random.choice(HEADERS_LIST)} to HEADER = {'User-Agent': random.choice(HEADERS_LIST), 'X-Requested-With': 'XMLHttpRequest'} but still have the same issue 'NoneType' object has no attribute 'user'.

Jun 03 '20 10:06 Frickson

I don't like modifying module's files directly, so instead of that and based on @rubengoeminne's great answer, to fix this issue you just have to add these line of codes to the top of your python script:

import twitterscraper
import random
HEADERS_LIST = [
    'Mozilla/5.0 (Windows; U; Windows NT 6.1; x64; fr; rv:1.9.2.13) Gecko/20101203 Firebird/3.6.13',
    'Mozilla/5.0 (compatible, MSIE 11, Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko',
    'Mozilla/5.0 (Windows; U; Windows NT 6.1; rv:2.2) Gecko/20110201',
    'Opera/9.80 (X11; Linux i686; Ubuntu/14.10) Presto/2.12.388 Version/12.16',
    'Mozilla/5.0 (Windows NT 5.2; RW; rv:7.0a1) Gecko/20091211 SeaMonkey/9.23a1pre'
]
twitterscraper.query.HEADER = {'User-Agent': random.choice(HEADERS_LIST), 'X-Requested-With': 'XMLHttpRequest'}

And do your stuff normally:

from twitterscraper import query_tweets
query_tweets("github", 100)

Jun 03 '20 13:06 javad94

This solution seems not to work for me now.

Jun 04 '20 01:06 Marlowe97

Yeah, unfortunately they close it down.

Jun 04 '20 01:06 javad94

guys are you sure that you replace the correct arquive? this is still working for me

Jun 04 '20 03:06 toscanopedro

@toscanopedro where did you replace please ? I replaced in the query.py file, and it's not working... Thanks !

Jun 04 '20 03:06 AlexBietrix

first you have to pip show twitterscraper to discover the locaticon of the twitterscraper directory. the mine was in: "c:\users\pedro\appdata\local\programs\python\python38-32\lib\site-packages" there is a folder called twitterscraper and the query.py arquive. And you have just to change it. the path may change, depends what idle you are using. but he is awais inside a "lib\site-packages"

Jun 04 '20 04:06 toscanopedro

@toscanopedro I am working on GCP, I changed the arquive manually as shown on the picture. Is is sufficient ? Capture d’écran 2020-06-03 à 21 14 04

Thanks

Jun 04 '20 04:06 AlexBietrix

twitterscraper twitterscraper copied to clipboard

0 tweets

twitterscraper
twitterscraper copied to clipboard