weiboSpider icon indicating copy to clipboard operation
weiboSpider copied to clipboard

'NoneType' object has no attribute 'xpath'报错

Open astarrrr opened this issue 2 years ago • 8 comments

为了更好的解决问题,请认真回答下面的问题。等到问题解决,请及时关闭本issue。

  • 问:请您指明哪个版本运行出错(github版/PyPi版/全部)?

答:github版

  • 问:您使用的是否是最新的程序(是/否)?

答:是

  • 问:爬取任意用户都会运行出错吗(是/否)?

答:是

  • 问:若只有爬特定微博时才出错,能否提供出错微博的weibo_id或url(非必填)?

答:

  • 问:若您已提供出错微博的weibo_id或url,可忽略此内容,否则能否提供出错账号的user_id及您配置的since_date,方便我们定位出错微博(非必填)?

答: "user_id_list": ["1699432410"], "filter": 1, "since_date": "2023-01-01", "end_date": "now", "random_wait_pages": [1, 3], "random_wait_seconds": [30, 40], "global_wait": [[1000, 3600], [500, 2000]],

  • 问:如果方便,请您描述出错详情,最好附上错误提示。

答:'NoneType' object has no attribute 'xpath' Traceback (most recent call last): File "C:\Users\astar\PycharmProjects\pythonProject5\weiboSpider\weibo_spider\spider.py", line 179, in get_weibo_info weibos, self.weibo_id_list, to_continue = PageParser( File "C:\Users\astar\PycharmProjects\pythonProject5\weiboSpider\weibo_spider\parser\page_parser.py", line 47, in init info = self.selector.xpath("//div[@class='c']") AttributeError: 'NoneType' object has no attribute 'xpath' 共爬取130条原创微博

astarrrr avatar Oct 24 '23 07:10 astarrrr

感谢反馈。可能是速度太快了,建议降低速度,过段时间再运行。

dataabc avatar Oct 24 '23 18:10 dataabc

感谢反馈。可能是速度太快了,建议降低速度,过段时间再运行 有没有建议的速度,还有除了改jason里的数值 还能怎么降低速度

astarrrr avatar Nov 01 '23 12:11 astarrrr

没有建议的速度,速度越慢越好,目前只能通过json设置。

dataabc avatar Nov 01 '23 17:11 dataabc

我也遭遇了这个错误,并且在json里调慢速度之后,依然每次在同一个位置停止抓取

shiverwang76 avatar Nov 10 '23 06:11 shiverwang76

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Mar 17 '24 12:03 stale[bot]

我也遇到同样的问题

xiaoyequ04 avatar Jun 02 '24 14:06 xiaoyequ04