twitterscraper icon indicating copy to clipboard operation
twitterscraper copied to clipboard

Why I am getting other than english language tweets??

Open VAC2018 opened this issue 4 years ago • 2 comments

Even after mentioning lang as english I am getting Hindi tweets

VAC2018 avatar Jun 09 '20 05:06 VAC2018

What command are you using to run?

lapp0 avatar Jun 09 '20 07:06 lapp0

I haven't used the lang attribute yet but instead have been using the langdetect library. It looks like something kind of like this.

`from langdetect import detect

def detector(s): try: return detect(s) except: None

tweets = query_tweets(query, begindate=begin_date, enddate=end_date, poolsize=1)

df = pd.DataFrame(t.__dict__ for t in tweets)

df['lang'] = df['text'].apply(lambda x: detector(x))
df = df[df['lang'] == 'en']

`

Sorry for the formatting

erb13020 avatar Jul 26 '20 19:07 erb13020