weibo-crawler icon indicating copy to clipboard operation
weibo-crawler copied to clipboard

该爬虫在爬取只有表情的微博时,会出现*****Error: Failed to extract text.

Open HeZhang1994 opened this issue 5 years ago • 5 comments

因为某些微博表情无法解码保存。

HeZhang1994 avatar Apr 21 '19 03:04 HeZhang1994

Traceback (most recent call last): File "run_WeiboCrawler.py", line 142, in cards_list.append(content['data']['cards']) # content['data']['cards'] KeyError: 'cards'

710898666 avatar Dec 04 '19 03:12 710898666

Traceback (most recent call last): File "run_WeiboCrawler.py", line 142, in cards_list.append(content['data']['cards']) # content['data']['cards'] KeyError: 'cards'

Hi, I did not get this error when I run this code today.

HeZhang1994 avatar Dec 04 '19 11:12 HeZhang1994

Traceback (most recent call last): File "run_WeiboCrawler.py", line 142, in cards_list.append(content['data']['cards']) # content['data']['cards'] KeyError: 'cards'

i got the same error as yours when i try this url USER_URL = r'https://m.weibo.cn/api/container/getIndex?type=uid&value=1722782045&containerid=1005051722782045'

little7Li avatar Feb 14 '20 04:02 little7Li

Traceback (most recent call last): File "run_WeiboCrawler.py", line 142, in cards_list.append(content['data']['cards']) # content['data']['cards'] KeyError: 'cards'

i got the same error as yours when i try this url USER_URL = r'https://m.weibo.cn/api/container/getIndex?type=uid&value=1722782045&containerid=1005051722782045'

i have solved this problem by correct the containerid in USER_URL, acctually when containerid = 107603+uid (such as when uid=1722782045 then containerid=1076031722782045 ), you can run this successfully!

little7Li avatar Feb 14 '20 06:02 little7Li

Traceback (most recent call last): File "run_WeiboCrawler.py", line 142, in cards_list.append(content['data']['cards']) # content['data']['cards'] KeyError: 'cards'

i got the same error as yours when i try this url USER_URL = r'https://m.weibo.cn/api/container/getIndex?type=uid&value=1722782045&containerid=1005051722782045'

i have solved this problem by correct the containerid in USER_URL, acctually when containerid = 107603+uid (such as when uid=1722782045 then containerid=1076031722782045 ), you can run this successfully!

Hi, thanks for your reply. I have tried to get the USER_URL of user 1722782045 according to the comment in the code. It should be https://m.weibo.cn/api/container/getIndex?type=uid&value=1722782045&containerid=1076031722782045 (just like yours). Maybe you can check the content in XHR/Headers, sometimes it will show alert or warning information. In this case, the URL you get might be wrong or invalid.

HeZhang1994 avatar Feb 14 '20 13:02 HeZhang1994