snscrape
snscrape copied to clipboard
`KeyError: 'mp4_720p_mp4'` on Weibo scrapes
When I attempt the following call (or any other weibo user)
snscrape --jsonl weibo-user 1223717857
I receive the following error:
2022-07-06 09:04:36.550 CRITICAL snscrape._cli Dumped stack and locals to /tmp/snscrape_locals_9a6n2i1t
Traceback (most recent call last):
File "/home/user/.local/bin/snscrape", line 8, in <module>
sys.exit(main())
File "/home/user/.local/lib/python3.8/site-packages/snscrape/_cli.py", line 318, in main
for i, item in enumerate(scraper.get_items(), start = 1):
File "/home/user/.local/lib/python3.8/site-packages/snscrape/modules/weibo.py", line 115, in get_items
yield self._mblog_to_item(card['mblog'])
File "/home/user/.local/lib/python3.8/site-packages/snscrape/modules/weibo.py", line 97, in _mblog_to_item
repostedPost = self._mblog_to_item(mblog['retweeted_status']) if 'retweeted_status' in mblog else None,
File "/home/user/.local/lib/python3.8/site-packages/snscrape/modules/weibo.py", line 95, in _mblog_to_item
video = mblog['page_info']['media_info']['mp4_720p_mp4'] if 'page_info' in mblog and mblog['page_info']['type'] == 'video' else None,
KeyError: 'mp4_720p_mp4'
Has anyone had any success scraping weibo lately?
Confirmed, thanks, I guess they changed the way videos are delivered.