twintproject
Can't find any tweet
Hi guys!
Can't find any tweets from user @BPharmCatalyst
[!] No more data! Scraping will stop now. found 0 deleted tweets in this search.
Could you check please:
twint -u BPharmCatalyst
Maybe it's just my mistake ((
Same for me.
twint uses Twitter search for scrape user's Tweets.
But @BPharmCatalyst's Tweets are not shown.
The cause may be Twitter's search rules.
Alternatively, you can scrape user's Tweets and RTs via twint -u BPharmCatalyst --retweets.
Only recent Tweets can be displayed.
This option will cause JSONDecodeError.
About the error, see #1335.
Same for me.
twint uses Twitter search for scrape user's Tweets. But
@BPharmCatalyst's Tweets are not shown. The cause may be Twitter's search rules.Alternatively, you can scrape user's Tweets and RTs via
twint -u BPharmCatalyst --retweets. Only recent Tweets can be displayed.This option will cause
JSONDecodeError. About the error, see #1335.
Many thanks!
I fixed the error JSONDecodeError as you said.
And now instead
twint.run.Search(c)
I use something like this to search for a tweets:
tweets = []
c = twint.Config()
c.Limit=100
c.Store_object_tweets_list = tweets
twint.run.Profile(c)
tweets = c.Store_object_tweets_list
p.s. Damn, twint needs a major update ((
@G-Pavel Hi,I failed to get tweets from user @BPharmCatalyst by your code ,Is there anything wrong with my use?
tweets = []
c = twint.Config()
c.Username = "BPharmCatalyst"
c.Limit=100
c.Store_object_tweets_list = tweets
twint.run.Profile(c)
tweets = c.Store_object_tweets_list```
@G-Pavel Hi,I failed to get tweets from user @BPharmCatalyst by your code ,Is there anything wrong with my use?
tweets = [] c = twint.Config() c.Username = "BPharmCatalyst" c.Limit=100 c.Store_object_tweets_list = tweets twint.run.Profile(c) tweets = c.Store_object_tweets_list```
Hi,
It twint -u BPharmCatalyst --retweets doesn't work anymore:
[!] Twitter does not return more data, scrape stops here.
I don't know what to do, it's complete nonsense (( @minamotorin do you or anyone else have any ideas?
@lyhtllw
You forgot c.Store_object = True.
# This patch is in the public domain. tweets = [] c = twint.Config() c.Username = "BPharmCatalyst" c.Limit=100 c.Store_object_tweets_list = tweets + c.Store_object = True twint.run.Profile(c) tweets = c.Store_object_tweets_list
@lyhtllw @G-Pavel The cause is Twitter's changes. I updated my code. minamotorin/twint ~~minamotorin/twint~~
This code looks working. But I haven't tested the code seriously. This means there might be omissions in the scraping results of Tweets (reports are welcome).
@lyhtllw You forgot
c.Store_object = True.# This patch is in the public domain. tweets = [] c = twint.Config() c.Username = "BPharmCatalyst" c.Limit=100 c.Store_object_tweets_list = tweets + c.Store_object = True twint.run.Profile(c) tweets = c.Store_object_tweets_list@lyhtllw @G-Pavel The cause is Twitter's changes. I updated my code. minamotorin/twint
This code looks working. But I haven't tested the code seriously. This means there might be omissions in the scraping results of Tweets (reports are welcome).
Seems to work, thanks a lot!
@lyhtllw You forgot
c.Store_object = True.# This patch is in the public domain. tweets = [] c = twint.Config() c.Username = "BPharmCatalyst" c.Limit=100 c.Store_object_tweets_list = tweets + c.Store_object = True twint.run.Profile(c) tweets = c.Store_object_tweets_list@lyhtllw @G-Pavel The cause is Twitter's changes. I updated my code. minamotorin/twint
This code looks working. But I haven't tested the code seriously. This means there might be omissions in the scraping results of Tweets (reports are welcome).
It's working, super helpful!!
This case is different, but it seems that the same problem may occur due to the account id changes.
E.g., if the id is changed from @before to @after, twint -u after return the tweets only after that point.
You can scrape the tweets until that point via twint -u before or twint -u after --retweets.
@lyhtllw You forgot
c.Store_object = True.# This patch is in the public domain. tweets = [] c = twint.Config() c.Username = "BPharmCatalyst" c.Limit=100 c.Store_object_tweets_list = tweets + c.Store_object = True twint.run.Profile(c) tweets = c.Store_object_tweets_list@lyhtllw @G-Pavel The cause is Twitter's changes. I updated my code. minamotorin/twint
This code looks working. But I haven't tested the code seriously. This means there might be omissions in the scraping results of Tweets (reports are welcome).
I tested the code. There are omissions in the searching results with c.Since and c.Until option. The number reported from the code mismatches with the number of tweets stored in csv. @minamotorin do you have any idea about this issue? Thanks.
@Tegrisco-ZSQ Are there any example? In my environment, the problem doesn't seems to be happening.
How can I see this number?
The number reported from the code
@Tegrisco-ZSQ Are there any example? In my environment, the problem doesn't seems to be happening.
How can I see this number?
The number reported from the code
@minamotorin The code I used:
def get_tweets(start_date, end_date, keyword, filename):
c = twint.Config()
c.Search = keyword
c.Since = start_date
c.Until = end_date
c.Hide_output = False
c.Lang = 'en'
c.Count = True
c.Store_csv = True
c.Output = filename
twint.run.Search(c)
get_tweets('2022-01-01', '2022-01-05', '$V','V.csv')
The keyword is a cashtag '$V'
The screenshot is the search result from code:
But I only got 317 tweets in the csv file. V.csv
This issue happens when I use other keywords like '$GME', '$MSFT', etc.
@Tegrisco-ZSQ
This can happen with also original twint and this has nothing to do with my code. Moreover, this is jut display bug and the mismatches doesn't mean there are omissions.
By the way, there is a another bug which causes omissions with since or until. #1136 Perhaps I fixed this in https://github.com/minamotorin/twint/commit/ef047337ad5daa60f98df55b14a68cb8045a928f, and the display bug may also be fixed as a result.
About bug of this update, please report to minamotorin/twint.