spider_collection
spider_collection copied to clipboard
[Bug]ip代理池为False, 配置cookie之后运行spider.py 报错 "用户回答获取失败,请带着以下报错去github原仓库提出issue"
具体项目 【e.g. 知乎用户爬虫】
bug描述 ip代理池为False, 配置cookie之后运行spider.py 报错 "用户回答获取失败,请带着以下报错去github原仓库提出issue"
报错信息 Traceback (most recent call last): File "C:\Users\10701\Downloads\zhihu_user_info_spider\zhihu_user_info_spider\requester\QuestionRequester.py", line 84, in __get_single_question_user_id total_num = self.__get_total(question) File "C:\Users\10701\Downloads\zhihu_user_info_spider\zhihu_user_info_spider\requester\QuestionRequester.py", line 118, in __get_total headers=self._header_with_zse_96(api_url=question_url), File "C:\Users\10701\Downloads\zhihu_user_info_spider\zhihu_user_info_spider\requester\ModeRequester.py", line 30, in header_with_zse_96 header["x-zse-96"] = Parser.get_x_zse_96(api_url) File "C:\Users\10701\Downloads\zhihu_user_info_spider\zhihu_user_info_spider\parser\Parser.py", line 130, in get_x_zse_96 encrypt_str = "2.0%s" % ctx1.call('b', fmd5) File "C:\Users\10701\Downloads\zhihu_user_info_spider\venv\lib\site-packages\execjs_abstract_runtime_context.py", line 37, in call return self._call(name, *args) File "C:\Users\10701\Downloads\zhihu_user_info_spider\venv\lib\site-packages\execjs_external_runtime.py", line 92, in _call return self.eval("{identifier}.apply(this, {args})".format(identifier=identifier, args=args)) File "C:\Users\10701\Downloads\zhihu_user_info_spider\venv\lib\site-packages\execjs_external_runtime.py", line 78, in eval return self.exec(code) File "C:\Users\10701\Downloads\zhihu_user_info_spider\venv\lib\site-packages\execjs_abstract_runtime_context.py", line 18, in exec return self.exec(source) File "C:\Users\10701\Downloads\zhihu_user_info_spider\venv\lib\site-packages\execjs_external_runtime.py", line 87, in exec output = self._exec_with_pipe(source) File "C:\Users\10701\Downloads\zhihu_user_info_spider\venv\lib\site-packages\execjs_external_runtime.py", line 99, in _exec_with_pipe p = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE, cwd=self._cwd, universal_newlines=True) File "D:\work\develop\Python\Python38\lib\subprocess.py", line 858, in init self._execute_child(args, executable, preexec_fn, close_fds, File "D:\work\develop\Python\Python38\lib\subprocess.py", line 1311, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, NotADirectoryError: [WinError 267] 目录名称无效。
系统环境
- 操作系统: Windows11
- ide: pycharm
- pyhon版本: 3..8