weibo-crawler icon indicating copy to clipboard operation
weibo-crawler copied to clipboard

windows下读取user_id_list配置文件路径不成功

Open Trisscute opened this issue 3 years ago • 2 comments

我将爬取列表的配置文件命名为user_id_list.txt,放在脚本的同级目录下,在配置文件中使用C:\Users\PC\Downloads\weibo-crawler-master和C:/Users/PC/Downloads/weibo-crawler-master以及user_id_list.txt均未能成功爬取,报错为提示被ban。以下是配置文件列表,我参照的是文档中7.定期自动爬取微博(可选)说明 { "user_id_list": ["C:\Users\PC\Downloads\weibo-crawler-master"], "filter": 1, "remove_html_tag": 1, "since_date": "2010-01-01", "start_page": 1, "write_mode": ["csv"], "original_pic_download": 1, "retweet_pic_download": 0, "original_video_download": 1, "retweet_video_download": 0, "download_comment":1, "comment_max_download_count":100, "result_dir_name": 0, "cookie": "", "mysql_config": { "host": "localhost", "port": 3306, "user": "root", "password": "123456", "charset": "utf8mb4" } }

Trisscute avatar Jan 05 '22 18:01 Trisscute

补充,我在将"user_id_list": ["C:\Users\PC\Downloads\weibo-crawler-master"],替换为"user_id_list": ["xxxxx"],单个userid之后能够正常爬取,似乎不是被ban了的原因?

Trisscute avatar Jan 05 '22 18:01 Trisscute

如果是路径,这样配置文件

"user_id_list":  "C:\Users\PC\Downloads\weibo-crawler-master\user_id_list.txt";

dataabc avatar Jan 06 '22 05:01 dataabc