weiboSpider icon indicating copy to clipboard operation
weiboSpider copied to clipboard

程序运行__main___出错

Open NocTurn294 opened this issue 1 year ago • 5 comments

为了更好的解决问题,请认真回答下面的问题。等到问题解决,请及时关闭本issue。

  • 问:请您指明哪个版本运行出错(github版/PyPi版/全部)?

答:python3.12

  • 问:您使用的是否是最新的程序(是/否)?

答:是

  • 问:爬取任意用户都会运行出错吗(是/否)?

答:是

  • 问:若只有爬特定微博时才出错,能否提供出错微博的weibo_id或url(非必填)?

答:否

  • 问:若您已提供出错微博的weibo_id或url,可忽略此内容,否则能否提供出错账号的user_id及您配置的since_date,方便我们定位出错微博(非必填)?

答:

  • 问:如果方便,请您描述出错详情,最好附上错误提示。

答:运行___main____.py 或者运行终端命令都会出现这个问题,

Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\parser\index_parser.py", line 37, in get_user
    self.user.weibo_num = string_to_int(user_info[0][3:-1])
                                        ~~~~~~~~~^^^
IndexError: list index out of range
None
****************************************************************************************************
'NoneType' object has no attribute 'nickname'
Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\spider.py", line 228, in _get_filepath
    dir_name = self.user.nickname
               ^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'nickname'
expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\writer\csv_writer.py", line 25, in __init__
    with open(self.file_path, 'a', encoding='utf-8-sig',
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not NoneType
****************************************************************************************************
Progress:   0%|          | 0/1 [00:02<?, ?it/s]
list index out of range
Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\parser\page_parser.py", line 66, in get_one_page
    is_exist = info[0].xpath("div/span[@class='ctt']")
               ~~~~^^^
IndexError: list index out of range
cannot unpack non-iterable NoneType object
Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\spider.py", line 180, in get_weibo_info
    weibos, self.weibo_id_list, to_continue = PageParser(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cannot unpack non-iterable NoneType object```

NocTurn294 avatar May 10 '24 00:05 NocTurn294

可能是爬取速度太快,被暂时限制了。一般情况下,一段时间后限制会自动解除。可通过降低爬取速度避免被限制。

dataabc avatar May 10 '24 06:05 dataabc

出现了一样的问题

MarthaWWW avatar May 14 '24 10:05 MarthaWWW

出现了一样的问题,降低爬取速度,隔了几天均无法进行爬取

QinXie123 avatar May 15 '24 05:05 QinXie123

更换在不同浏览器的cookie就行,edge的cookie不行就换chrome,亲测有用

MarthaWWW avatar May 16 '24 03:05 MarthaWWW

出现了一样的问题

xiaoyequ04 avatar Jun 02 '24 14:06 xiaoyequ04

the same problem popped up again

YUTING0907 avatar Dec 25 '24 09:12 YUTING0907

更换在不同浏览器的cookie就行,edge的cookie不行就换chrome,亲测有用 我试过了 还是不行 o(╥﹏╥)o

YUTING0907 avatar Dec 25 '24 09:12 YUTING0907