snscrape
snscrape copied to clipboard
Crash on `TweetUnavailable` results
snscrape twitter-tweet --recurse 1500507885922619394
Traceback (most recent call last):
File "/home/thetechrobo/.local/bin/snscrape", line 8, in <module>
sys.exit(main())
File "/home/thetechrobo/.local/lib/python3.9/site-packages/snscrape/_cli.py", line 308, in main
for i, item in enumerate(scraper.get_items(), start = 1):
File "/home/thetechrobo/.local/lib/python3.9/site-packages/snscrape/modules/twitter.py", line 1030, in get_items
for tweet in self._graphql_timeline_instructions_to_tweets(obj['data']['threaded_conversation_with_injections']['instructions'], includeConversationThreads = True):
File "/home/thetechrobo/.local/lib/python3.9/site-packages/snscrape/modules/twitter.py", line 688, in _graphql_timeline_instructions_to_tweets
yield self._graphql_timeline_tweet_item_result_to_tweet(item['item']['itemContent']['tweet_results']['result'])
File "/home/thetechrobo/.local/lib/python3.9/site-packages/snscrape/modules/twitter.py", line 652, in _graphql_timeline_tweet_item_result_to_tweet
raise snscrape.base.ScraperException(f'Unknown result type {result["__typename"]!r}')
snscrape.base.ScraperException: Unknown result type 'TweetUnavailable'
Dump: https://transfer.archivete.am/60upN/stack
I've seen this before during my card code tests, but I have no idea yet what causes it. The web interface simply doesn't show the requested tweet at all in these cases. It seems somewhat related to tombstones (#392) per Twitter's JS, but the returned data has zero context details, unlike proper tombstones.
Two of the IDs causing it in this particular example: 1500514518631387140 1500529138691710987
At least those two examples are showing as age-restricted now (cf. #419).
Yeah, I noticed that too earlier today.
I think I found a reliable way of reproducing this while implementing communities (#614): 18+ posts in communities appear to always be returned as TweetUnavailable
. Community 1560462400503296001 has one of those.