MediaCrawler icon indicating copy to clipboard operation
MediaCrawler copied to clipboard
Published 6 hours ago • verified publisher icon NanmiCoder

[问题] 程序运行扫码登陆完成后自动关闭了页面

Open Ellie-xi opened this issue 5 months ago • 9 comments

⚠️ 提交前确认

  • [ ] 我已经仔细阅读了项目使用过程中的常见问题汇总
  • [ ] 我已经搜索并查看了已关闭的issues
  • [ ] 我确认这不是由于滑块验证码、Cookie过期、Cookie提取错误、平台风控等常见原因导致的问题

❓ 问题描述

🔍 使用场景

  • 目标平台: (如：小红书/抖音/微博等) 小红书
  • 使用功能: (如：关键词搜索/用户主页爬取等) 关键词搜索

💻 环境信息

  • 操作系统: win11
  • Python版本: 3.12
  • 是否使用IP代理: 否
  • 是否使用VPN翻墙软件：是
  • 目标平台(抖音/小红书/微博等): 小红书

📋 错误日志

shell PS D:\爬虫\1\MediaCrawler> python main.py C:\Users\chxii\AppData\Roaming\Python\Python312\site-packages\jieba_compat.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. import pkg_resources 2025-07-21 00:19:27 MediaCrawler INFO (core.py:75) - [XiaoHongShuCrawler] 使用CDP模式启动浏览器 2025-07-21 00:19:27 MediaCrawler INFO (cdp_browser.py:94) - [CDPBrowserManager] 检测到浏览器: Google Chrome (正在现有的浏览器会话中打开。) 2025-07-21 00:19:27 MediaCrawler INFO (cdp_browser.py:97) - [CDPBrowserManager] 浏览器路径: C:\Program Files\Google\Chrome\Application\chrome.exe 2025-07-21 00:19:27 MediaCrawler INFO (cdp_browser.py:137) - [CDPBrowserManager] 用户数据目录: D:\爬虫\1\MediaCrawler\browser_data\cdp_xhs_user_data_dir 2025-07-21 00:19:27 MediaCrawler INFO (browser_launcher.py:152) - [BrowserLauncher] 启动浏览器: C:\Program Files\Google\Chrome\Application\chrome.exe 2025-07-21 00:19:27 MediaCrawler INFO (browser_launcher.py:153) - [BrowserLauncher] 调试端口: 9222 2025-07-21 00:19:27 MediaCrawler INFO (browser_launcher.py:154) - [BrowserLauncher] 无头模式: False 2025-07-21 00:19:27 MediaCrawler INFO (browser_launcher.py:183) - [BrowserLauncher] 等待浏览器在端口 9222 上准备就绪... 2025-07-21 00:19:27 MediaCrawler INFO (browser_launcher.py:192) - [BrowserLauncher] 浏览器已在端口 9222 上准备就绪 2025-07-21 00:19:28 MediaCrawler INFO (cdp_browser.py:111) - [CDPBrowserManager] CDP端口 9222 可访问 2025-07-21 00:19:29 MediaCrawler INFO (cdp_browser.py:175) - [CDPBrowserManager] 获取到浏览器WebSocket URL: ws://localhost:9222/devtools/browser/b02be684-aea9-4fd3-b04e-18a062c35cc8 2025-07-21 00:19:29 MediaCrawler INFO (cdp_browser.py:194) - [CDPBrowserManager] 正在通过CDP连接到浏览器: ws://localhost:9222/devtools/browser/b02be684-aea9-4fd3-b04e-18a062c35cc8 2025-07-21 00:19:29 MediaCrawler INFO (cdp_browser.py:200) - [CDPBrowserManager] 成功连接到浏览器 2025-07-21 00:19:29 MediaCrawler INFO (cdp_browser.py:201) - [CDPBrowserManager] 浏览器上下文数量: 1 2025-07-21 00:19:29 MediaCrawler INFO (cdp_browser.py:226) - [CDPBrowserManager] 使用现有的浏览器上下文 2025-07-21 00:19:29 MediaCrawler INFO (core.py:503) - [XiaoHongShuCrawler] CDP浏览器信息: {'version': '138.0.7204.101', 'contexts_count': 1, 'debug_port': 9222, 'is_connected': True} 2025-07-21 00:19:31 MediaCrawler INFO (core.py:429) - [XiaoHongShuCrawler.create_xhs_client] Begin create xiaohongshu API client ... 2025-07-21 00:19:31 MediaCrawler INFO (client.py:176) - [XiaoHongShuClient.pong] Begin to pong xhs... 2025-07-21 00:19:35 MediaCrawler ERROR (client.py:183) - [XiaoHongShuClient.pong] Ping xhs failed: RetryError[<Future at 0x20d64884ec0 state=finished raised DataFetchError>], and try to login again... 2025-07-21 00:19:35 MediaCrawler INFO (login.py:62) - [XiaoHongShuLogin.begin] Begin login xiaohongshu ... 2025-07-21 00:19:35 MediaCrawler INFO (login.py:142) - [XiaoHongShuLogin.login_by_qrcode] Begin login xiaohongshu by qrcode ... 2025-07-21 00:19:35 MediaCrawler INFO (login.py:175) - [XiaoHongShuLogin.login_by_qrcode] waiting for scan code login, remaining time is 120s 2025-07-21 00:19:36 MediaCrawler INFO (login.py:183) - [XiaoHongShuLogin.login_by_qrcode] Login successful then wait for 5 seconds redirect ... 2025-07-21 00:19:41 MediaCrawler INFO (core.py:140) - [XiaoHongShuCrawler.search] Begin search xiaohongshu keywords 2025-07-21 00:19:41 MediaCrawler INFO (core.py:149) - [XiaoHongShuCrawler.search] Current search keyword: 心血管疾病 2025-07-21 00:19:41 MediaCrawler INFO (core.py:163) - [XiaoHongShuCrawler.search] search xhs keyword: 心血管疾病, page: 1
2025-07-21 00:19:43 MediaCrawler INFO (core.py:178) - [XiaoHongShuCrawler.search] Search notes res:{'has_more': True, 'items': [{'id': '65b76640000000002b03ff73', 'model_type': 'note', 'note_card': {'type': 'normal', 'display_title': '好医保重磅 新品！最长可保终身的百万医疗险', 'user': {'user_id': '618a50890000000010009dbf', 'nickname': '蚂蚁保-好医保', 'xsec_token': 'ABYJVCnNfZlmpV7ribd0q2-Rx7CQjiVUEU8XBrjloi59I=', 'nick_name': '蚂蚁保-好医保', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo318nkq12ljq005ocaa24k17dvnbcthvo?imageView2/2/w/80/format/jpg'}, 'interact_info': {'collected_count': '12132', 'comment_count': '2060', 'shared_count': '2603', 'liked': False, 'liked_count': '12906', 'collected': False}, 'cover': {'width': 1242, 'height': 1660}, 'image_list': [{'width': 1242, 'height': 1660}, {'height': 1920, 'width': 1440}, {'height': 1660, 'width': 1242}, {'height': 1920, 'width': 1440}, {'width': 1242, 'height': 1660}, {'height': 1660, 'width': 1242}, {'height': 1920, 'width': 1440}, {'width': 1242, 'height': 1660}], 'corner_tag_info': [{'type': 'publish_time', 'text': '2024-10-21'}]}, 'xsec_token': 'ABplwkZr6SZvVUyWvian7PgITlg76I0knotanmCTg1iLk='}, {'id': '66a9ace2000000000600db26', 'model_type': 'note', 'note_card': {'image_list': [{'height': 2880, 'width': 2160}], 'corner_tag_info': [{'type': 'publish_time', 'text': '2024-07-31'}], 'type': 'video', 'display_title': '学会了能提高心梗自救成功率！', 'user': {'nick_name': '丁香医生', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/60a72679930bba000149e57a.jpg?imageView2/2/w/80/format/jpg', 'user_id': '5bab2e53a57b6b0001d6d9d6', 'nickname': '丁香医生', 'xsec_token': 'ABsuMWZUFQin0cCS37GQx_rT9ERSAU2uWCzcLDFBC7-JY='}, 'interact_info': {'collected': False, 'collected_count': '2486', 'comment_count': '297', 'shared_count': '2111', 'liked': False, 'liked_count': '4257'}, 'cover': {'height': 2880, 'width': 2160}}, 'xsec_token': 'ABd-ToLcGN7Y8ydlEAqNPKwaWZEO37PYd1cV8FAj9_65w='}, {'note_card': {'image_list': [{'height': 2560, 'width': 1920}], 'corner_tag_info': [{'type': 'publish_time', 'text': '6天前'}], 'type': 'video', 'display_title': '你以为问题不大，实际上很严重的疾病表现！', 'user': {'nick_name': '心内科黄 宇靖医生', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo318ami17f0q605n2pjpa4cv8l0tbr0u8?imageView2/2/w/80/format/jpg', 'user_id': '5c599e540000000011027d15', 'nickname': '心内科黄宇靖医生', 'xsec_token': 'ABmvsHm_i8esd7ykTE4_XXiBbWSjHyRIMl3BxGI_WaoxI='}, 'interact_info': {'collected': False, 'collected_count': '651', 'comment_count': '170', 'shared_count': '59', 'liked': False, 'liked_count': '2399'}, 'cover': {'height': 2560, 'width': 1920}}, 'xsec_token': 'ABEOSw-sBSDUCuZBb-fln6jYm32Yrzxr90gFTdauWhesw=', 'id': '6874e31d000000000b02ff61', 'model_type': 'note'}, {'id': '684902c90000000023016133', 'model_type': 'note', 'note_card': {'display_title': '多吃清理血管的10种食物', 'user': {'nickname': 'Flora爱营养', 'xsec_token': 'ABWAv4N5Ey0wMoFyLRexm-rLibzMX70iAYT4LnxxnzB7I=', 'nick_name': 'Flora爱营养', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo31iivendpni6g5pnajrn2u3o9ms81ihg?imageView2/2/w/80/format/jpg', 'user_id': '66ea9eee000000000b030f09'}, 'interact_info': {'liked': False, 'liked_count': '2388', 'collected': False, 'collected_count': '2920', 'comment_count': '50', 'shared_count': '1287'}, 'cover': {'height': 2730, 'width': 2048}, 'image_list': [{'height': 2730, 'width': 2048}, {'height': 2730, 'width': 2048}, {'height': 2730, 'width': 2048}, {'width': 2048, 'height': 2730}, {'height': 2730, 'width': 2048}, {'height': 2730, 'width': 2048}], 'corner_tag_info': [{'type': 'publish_time', 'text': '06-11'}], 'type': 'normal'}, 'xsec_token': 'ABj1FCjHLQdyu1H_92ZdlZYLGc1xpRcgG1mN3oGCfO3Ds='}, {'id': '68719ba5000000001100042c', 'model_type': 'note', 'note_card': {'interact_info': {'shared_count': '775', 'liked': False, 'liked_count': '1132', 'collected': False, 'collected_count': '1465', 'comment_count': '182'}, 'cover': {'width': 1200, 'height': 1600}, 'image_list': [{'height': 1600, 'width': 1200}, {'height': 1920, 'width': 1440}, {'height': 1920, 'width': 1440}, {'height': 1920, 'width': 1440}, {'height': 1356, 'width': 987}], 'corner_tag_info': [{'type': 'publish_time', 'text': '6天前'}], 'type': 'normal', 'display_title': ' 颈动脉斑块从查出到消失，1年里我做了什么', 'user': {'nick_name': '斑块消消日记', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/53ba1722242c3e9344db7b0fc2e1005f.jpg?imageView2/2/w/80/format/jpg', 'user_id': '5f45925a00000000010040da', 'nickname': '斑块消消日记', 'xsec_token': 'ABRK6Y1zKEnj8ns4ijhaF5DeJDOBtKgnbIoiVPAsCmzoo='}}, 'xsec_token': 'ABn9buOHclmnVqp8vwiLtqwTr7Lz0n75q--4dEK2_Gobs='}, {'xsec_token': 'AByZrxSYtBdfpxodnwvdBwzxV2Sv89CMzio2BeDEHm8JA=', 'id': '680324da000000000b02d51b', 'model_type': 'note', 'note_card': {'image_list': [{'height': 1600, 'width': 1200}], 'corner_tag_info': [{'type': 'publish_time', 'text': '05-15'}], 'type': 'normal', 'display_title': '国家喊停！卫健委点名3大危险减肥法', 'user': {'nick_name': '君君减脂记', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo316v68id73u6g5pm9tm8nebdbjk2ug8g?imageView2/2/w/80/format/jpg', 'user_id': '66c9ed91000000001d032dab', 'nickname': '君君减脂记', 'xsec_token': 'AB9MBwx30omeTxwYDtyYXKt3kuEqWUmJjUr9_waLEcju4='}, 'interact_info': {'collected': False, 'collected_count': '648', 'comment_count': '209', 'shared_count': '601', 'liked': False, 'liked_count': '959'}, 'cover': {'width': 1200, 'height': 1600}}}, {'xsec_token': 'ABwwMn4xMJArZITdYd7mv0RB02K0yAkPoXSKpRHndWroM=', 'id': '68076d72000000001c035234', 'model_type': 'note', 'note_card': {'type': 'normal', 'display_title': '胸闷气短、极度疲乏第一步：全面体检', 'user': {'nickname': '中年七的健康自救笔记', 'xsec_token': 'ABbqs23eQpWK6pQgY8JNRGaFw3VhchbtbGfztHqZqZPus=', 'nick_name': '中年七的健康自救笔记', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo319duuns3lo005n763d25p08iier1uh0?imageView2/2/w/80/format/jpg', 'user_id': '5ce61b440000000017008112'}, 'interact_info': {'shared_count': '183', 'liked': False, 'liked_count': '934', 'collected': False, 'collected_count': '860', 'comment_count': '79'}, 'cover': {'width': 1200, 'height': 1600}, 'image_list': [{'height': 1600, 'width': 1200}], 'corner_tag_info': [{'text': '04-22', 'type': 'publish_time'}]}}, {'id': '683a8b450000000021004ba9', 'model_type': 'note', 'note_card': {'cover': {'height': 815, 'width': 1206}, 'image_list': [{'height': 815, 'width': 1206}], 'corner_tag_info': [{'type': 'publish_time', 'text': '05-31'}], 'type': 'normal', 'display_title': '父母心血管问题得出的感想', 'user': {'nickname': '多晒 太阳🌞', 'xsec_token': 'ABS0qrpe4GwAHQfKXmCMzgaEsThXGtPP9061xXolMKbTI=', 'nick_name': '多晒太阳🌞', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo31i61lt4i7a004a3nmd0icrn1josmvc8?imageView2/2/w/80/format/jpg', 'user_id': '5aaf8126e8ac2b1e4b3e6ee1'}, 'interact_info': {'collected_count': '801', 'comment_count': '162', 'shared_count': '102', 'liked': False, 'liked_count': '930', 'collected': False}}, 'xsec_token': 'ABoRI2Yl7mUoJ6ZB9dhUynfQD8TSF67B9HXrubGv_dabY='}, {'model_type': 'note', 'note_card': {'display_title': '心血管死亡风险↓31%…来自瑞典人的Fika建议', 'user': {'nick_name': 'Astaxin', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/65253e49e0cc0eb380956bf2.jpg?imageView2/2/w/80/format/jpg', 'user_id': '5f37d6130000000001008ef8', 'nickname': 'Astaxin', 'xsec_token': 'AB4qqYk2ZoFU7nl_UWrIr2KGdbiQMxWGOnTgNac_iyCQI='}, 'interact_info': {'collected_count': '520', 'comment_count': '166', 'shared_count': '1126', 'liked': False, 'liked_count': '874', 'collected': False}, 'cover': {'height': 1280, 'width': 960}, 'image_list': [{'height': 1280, 'width': 960}, {'height': 1280, 'width': 960}, {'height': 1280, 'width': 960}, {'height': 1280, 'width': 960}, {'height': 1280, 'width': 960}], 'corner_tag_info': [{'type': 'publish_time', 'text': '03-13'}], 'type': 'normal'}, 'xsec_token': 'ABqrn9rmNKUNt2IFAeOrx5RVL8--tJg4J-jaIqWVtPqPQ=', 'id': '67cfcf7700000000290357f5'}, {'id': 'c85b83ca-25ed-4994-9798-6fd7619c14cb#1753028383220', 'model_type': 'rec_query', 'rec_query': {'word_request_id': 'c85b83ca-25ed-4994-9798-6fd7619c14cb#1753028383220', 'queries': [{'name': '心血 管疾病有哪些', 'search_word': '心血管疾病有哪些', 'id': '心血管疾病有哪些'}, {'id': '心血管疾病类型', 'name': '心血管疾病类型', 'search_word': '心血管疾病类型'}, {'id': '心血管疾病治疗', 'name': '心血管疾病治疗', 'search_word': '心血管疾病治疗'}, {'id': '常见的心血管疾病', 'name': '常见的心血管疾病', 'search_word': '常见的心血管疾病'}], 'title': '相关搜索', 'source': 1}, 'xsec_token': 'ABfEwGrTL2BL5wil-3c9xZOfyVMSfb6gn7C7-Nqkk2qKIIw3IYZh4dEvonDl-i_mvopBmCM4FJvr5kjaOz57H-Mu5t5jSKCJiR2K-sgxmHosY='}, {'id': '65488e8200000000250148fe', 'model_type': 'note', 'note_card': {'cover': {'height': 2560, 'width': 1920}, 'image_list': [{'height': 2560, 'width': 1920}, {'height': 2560, 'width': 1920}, {'height': 2560, 'width': 1920}, {'height': 2560, 'width': 1920}, {'height': 2560, 'width': 1920}, {'height': 2560, 'width': 1920}, {'height': 2276, 'width': 1280}], 'corner_tag_info': [{'type': 'publish_time', 'text': '2023-11-07'}], 'type': 'normal', 'display_title': '《黄帝内经》讲血管病跟心有关', 'user': {'nick_name': '大麦的养生之道', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1000g2jo2nc4917cju06g5nvvls60bq5a4s7kra0?imageView2/2/w/80/format/jpg', 'user_id': '5fffaf0c000000000101e8aa', 'nickname': '大麦的养生之道', 'xsec_token': 'ABCkh_9Ap8d9geInvABmo5AxS4qik2prInpVt8WWBuxu4='}, 'interact_info': {'liked_count': '583', 'collected': False, 'collected_count': '356', 'comment_count': '6', 'shared_count': '29', 'liked': False}}, 'xsec_token': 'ABelPSVuukJPsxOSE6UCxwGtnevcu2NIaM6Yg8qOUlhb0='}, {'xsec_token': 'AB77eUNoFAJhNrQQnpuTzkAJjlkbUahwTMDCSsO5ipDOo=', 'id': '6867a83500000000170377ae', 'model_type': 'note', 'note_card': {'user': {'nick_name': '红薯科研人', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo31hhhku2b3qdg5q141q7jibqsnuplapg?imageView2/2/w/80/format/jpg', 'user_id': '68240e8f000000000e012f5c', 'nickname': '红薯科研人', 'xsec_token': 'AB17w2hAqnYKZwGBWnHweqc6b-qFiti5ZQTGJ4R2w4FKo='}, 'interact_info': {'collected_count': '636', 'comment_count': '7', 'shared_count': '100', 'liked': False, 'liked_count': '568', 'collected': False}, 'cover': {'height': 1600, 'width': 1200}, 'image_list': [{'height': 1600, 'width': 1200}], 'corner_tag_info': [{'type': 'publish_time', 'text': '07-04'}], 'type': 'normal', 'display_title': '下一个发文新风口：检验科共病 赢麻了'}}, {'id': '67f736f5000000001b03ae15', 'model_type': 'note', 'note_card': {'cover': {'height': 1600, 'width': 1200}, 'image_list': [{'height': 1600, 'width': 1200}], 'corner_tag_info': [{'text': '04-10', 'type': 'publish_time'}], 'type': 'normal', 'display_title': '钟南山紧 急提醒：可致突然si亡！', 'user': {'nick_name': '迈基诺林', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo31jltfcvrj25g5pr94ga62k5v5mc8bhg?imageView2/2/w/80/format/jpg', 'user_id': '6769241400000000180150bf', 'nickname': '迈基诺林', 'xsec_token': 'AB0rwosL9DGdxqdo_lFgUHk1QMIh7WNlS9OEK2XxUWD5g='}, 'interact_info': {'comment_count': '8', 'shared_count': '112', 'liked': False, 'liked_count': '495', 'collected': False, 'collected_count': '201'}}, 'xsec_token': 'ABZPU9okGfhPbj31XYpdc6t45f9dA0s6k7yrE0hxh_SKY='}, {'xsec_token': 'AB2Tqdm8Kzau3HS5hS27usP0ofMgvusrVwzKOjJKBl2g8=', 'id': '670b9f1500000000210011c5', 'model_type': 'note', 'note_card': {'cover': {'height': 2560, 'width': 1920}, 'image_list': [{'height': 2560, 'width': 1920}, {'height': 1920, 'width': 1440}, {'height': 1920, 'width': 1440}, {'height': 1920, 'width': 1440}, {'width': 1440, 'height': 1920}, {'height': 1920, 'width': 1440}, {'height': 1920, 'width': 1440}, {'height': 1920, 'width': 1440}, {'width': 1440, 'height': 1920}, {'height': 1920, 'width': 1440}], 'corner_tag_info': [{'type': 'publish_time', 'text': '2024-10-13'}], 'type': 'normal', 'display_title': '🎧 一些最近在听的全英播客（健康科普向）', 'user': {'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/6691249e28ae847124b501ec.jpg?imageView2/2/w/80/format/jpg', 'user_id': '6522abfa000000002b003b74', 'nickname': '阿Pe你又买啥了', 'xsec_token': 'ABdlDuOy659hKQ2A37EUA4oGKq0NWQMqO3eFO4C_n68p4=', 'nick_name': '阿Pe你又买啥了'}, 'interact_info': {'collected_count': '315', 'comment_count': '14', 'shared_count': '33', 'liked': False, 'liked_count': '497', 'collected': False}}}, {'xsec_token': 'ABlBg18gc5nItMBQfi6ZysBSrsWjvlOec1aqBlu1aAYrw=', 'id': '666c0bcc000000001d01ad97', 'model_type': 'note', 'note_card': {'type': 'normal', 'display_title': '一文解读：心脑血管疾病之“血栓”', 'user': {'nick_name': 'ZINGY紫一', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/66026d6ea81d5d0001e5216a.jpg?imageView2/2/w/80/format/jpg', 'user_id': '6269e3f6000000001000ad79', 'nickname': 'ZINGY紫一', 'xsec_token': 'AB554PRgYzejWhG9tMNQj1UYCueXIpnN_fCw7v5YgfA5s='}, 'interact_info': {'collected_count': '345', 'comment_count': '16', 'shared_count': '282', 'liked': False, 'liked_count': '469', 'collected': False}, 'cover': {'height': 1000, 'width': 750}, 'image_list': [{'height': 1000, 'width': 750}, {'height': 2560, 'width': 1920}, {'height': 1000, 'width': 750}, {'height': 2560, 'width': 1920}], 'corner_tag_info': [{'text': '2024-06-14', 'type': 'publish_time'}]}}, {'id': '66fa0d6c000000002c02d308', 'model_type': 'note', 'note_card': {'interact_info': {'comment_count': '4', 'shared_count': '102', 'liked': False, 'liked_count': '396', 'collected': False, 'collected_count': '221'}, 'cover': {'height': 1632, 'width': 1242}, 'image_list': [{'height': 1632, 'width': 1242}, {'height': 1632, 'width': 1242}, {'height': 1632, 'width': 1242}, {'height': 1632, 'width': 1242}, {'height': 1632, 'width': 1242}, {'height': 1632, 'width': 1242}], 'corner_tag_info': [{'type': 'publish_time', 'text': '2024-10-03'}], 'type': 'normal', 'display_title': '四种常见心血管疾病及其症状和预防方法', 'user': {'nick_name': '瑞枫盛达-医疗器械', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo311vdbq9r1o605pa7n7k38kvm0q88kmg?imageView2/2/w/80/format/jpg', 'user_id': '6547b9e8000000000d0053f6', 'nickname': '瑞枫盛达-医疗器械', 'xsec_token': 'AB9l744exTcs3VpJcOUc08X7qIKb8_x0EHp9A7XCCF100='}}, 'xsec_token': 'ABWeWLJ-yAtULiVo40UFGhuZgh9PGbfb_p5KaMJJOcBSU='}, {'xsec_token': 'ABaMyAERcnp-nc8UNWmWuwd3gGIqq4QNl5-HH-v1raFaQ=', 'id': '656de160000000000703ad3e', 'model_type': 'note', 'note_card': {'user': {'nick_name': '爱分享的晓煜Carol姐', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo31elov0dclg0g5nomtqv08iicmcu6mg8?imageView2/2/w/80/format/jpg', 'user_id': '5f16eebe0000000001004a4c', 'nickname': '爱分享的晓煜Carol姐', 'xsec_token': 'ABuio9l2RodeOxEdEFv_G1PeiN46bCdvJ15qBR7naLc9Q='}, 'interact_info': {'liked': False, 'liked_count': '393', 'collected': False, 'collected_count': '375', 'comment_count': '8', 'shared_count': '860'}, 'cover': {'height': 1280, 'width': 720}, 'image_list': [{'height': 1280, 'width': 720}], 'corner_tag_info': [{'text': '2023-12-04', 'type': 'publish_time'}], 'type': 'video', 'display_title': '血管垃圾不及时清理 ，会诱发心脑血管疾病'}}, {'id': '66755e17000000001d01470f', 'model_type': 'note', 'note_card': {'type': 'normal', 'display_title': '伟哥真的治疗心血管疾病吗', 'user': {'nick_name': '柒同学', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo311k3t3kr6o605pgt3df1ogdupjkf4po?imageView2/2/w/80/format/jpg', 'user_id': '661d1cd5000000000700557a', 'nickname': '柒 同学', 'xsec_token': 'ABtB3ZNNp-1_2JbikBRS78j3cWqtKsCM4tKPxDcrHI-RI='}, 'interact_info': {'shared_count': '314', 'liked': False, 'liked_count': '385', 'collected': False, 'collected_count': '86', 'comment_count': '57'}, 'cover': {'width': 1284, 'height': 1652}, 'image_list': [{'width': 1284, 'height': 1652}, {'width': 1284, 'height': 1661}], 'corner_tag_info': [{'type': 'publish_time', 'text': '2024-06-21'}]}, 'xsec_token': 'ABjWOrMls7phMSiVvk24VsTxsK5Wococv0yeX5EmvzjQU='}, {'id': '678344890000000020015cfc', 'model_type': 'note', 'note_card': {'corner_tag_info': [{'type': 'publish_time', 'text': '01-12'}], 'type': 'normal', 'display_title': 'Adv Mater｜可穿戴柔性电子&中医号脉', 'user': {'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo318rj3ph10o005nrmhcv0bvl542i3tk8?imageView2/2/w/80/format/jpg', 'fEwGrTL2BL5wil-3c9xZOfyVMSfb6gn7C7-Nqkk2qKIIw3IYZh4dEvonDl-i_mvopBmCM4FJvr5kjaOz57H-Mu5t5jSKCJiR2K-sgxmHosY='}, {'model_type': 'note', 'note_card': {'type': 'video', 'display_title': '', 'user': {'user_id': '66c47809000000001d02132c', 'nickname': '心血管张蕴慧教授', 'xsec_token': 'ABBkQwH0sAZ-ousuf7_nTiu2HtpEnV6xJrpfqUSJLb_Vk=', 'nick_name': '心血管张蕴慧教授', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo316v5f9sdgo6g5pm4f04nc4pcvvg1fpg?imageView2/2/w/80/format/jpg'}, 'interact_info': {'shared_count': '253', 'liked': False, 'liked_count': '343', 'collected': False, 'collected_count': '513', 'comment_count': '15'}, 'cover': {'height': 3840, 'width': 2160}, 'image_list': [{'height': 3840, 'width': 2160}], 'corner_tag_info': [{'type': 'publish_time', 'text': '04-22'}]}, 'xsec_token': 'ABwwMn4xMJArZITdYd7mv0RO0bFUTK_7I51hHdXzLbN6k=', 'id': '6807293e000000001d00608b'}, {'id': '675d59e2000000000103d01e', 'model_type': 'note', 'note_card': {'image_list': [{'height': 884, 'width': 1572}], 'corner_tag_info': [{'type': 'publish_time', 'text': '2024-12-14'}], 'type': 'video', 'display_title': '🫀高血压|心血管常见疾病之一', 'user': {'nick_name': '栗子超级nice😻', 'avatar': 'https://sns-avatar-qc.xhscdn.com/avatar/1040g2jo30qiq6f2h0c005nsg4ak08jg137km038?imageView2/2/w/80/format/jpg', 'user_id': '5f9022a80000000001004e01', 'nickname': '栗子超级nice😻', 'xsec_token': 'ABXZpN1OsUkiHnRLYRhBB0XOLOgN2K759bvq7C66Eo-74='}, 'interact_info': {'liked_count': '279', 'collected': False, 'collected_count': '205', 'comment_count': '42', 'shared_count': '384', 'liked': False}, 'cover': {'height': 884, 'width': 1572}}, 'xsec_token': 'ABpZXFenR7MyDf9At4YeouwfMNywD7Z3G39-DHinjscxQ='}]} 2025-07-21 00:19:43 MediaCrawler INFO (core.py:322) - [get_note_detail_async_task] Begin get note detail, note_id: 65b76640000000002b03ff73 2025-07-21 00:19:47 MediaCrawler ERROR (core.py:339) - [XiaoHongShuCrawler.get_note_detail_async_task] Get note detail error, note_id: 65b76640000000002b03ff73 2025-07-21 00:19:52 MediaCrawler INFO (core.py:322) - [get_note_detail_async_task] Begin get note detail, note_id: 66a9ace2000000000600db26 2025-07-21 00:19:52 MediaCrawler INFO (browser_launcher.py:235) - [BrowserLauncher] 正在关闭浏览器进程... 2025-07-21 00:19:53 MediaCrawler INFO (browser_launcher.py:246) - [BrowserLauncher] 浏览器进程已关闭 2025-07-21 00:19:53 MediaCrawler INFO (core.py:525) - [XiaoHongShuCrawler.close] Browser context closed ... Traceback (most recent call last): File "C:\Users\chxii\AppData\Roaming\Python\Python312\site-packages\tenacity\asyncio_init_.py", line 114, in call result = await fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\爬虫\1\MediaCrawler\media_platform\xhs\client.py", line 106, in request raise Exception( Exception: 出现验证码，请求失败，Verifytype: 124，Verifyuuid: 10a5c41c-af77-401e-8131-1850e951a92d, Response: <Response [461 status code 461]>

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "D:\爬虫\1\MediaCrawler\main.py", line 77, in asyncio.get_event_loop().run_until_complete(main()) File "D:\Program Files\Python312\Lib\asyncio\base_events.py", line 687, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "D:\爬虫\1\MediaCrawler\main.py", line 65, in main await crawler.start() File "D:\爬虫\1\MediaCrawler\media_platform\xhs\core.py", line 126, in start await self.search() File "D:\爬虫\1\MediaCrawler\media_platform\xhs\core.py", line 195, in search note_details = await asyncio.gather(*task_list) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\爬虫\1\MediaCrawler\media_platform\xhs\core.py", line 345, in get_note_detail_async_task await self.xhs_client.get_note_by_id( File "D:\爬虫\1\MediaCrawler\media_platform\xhs\client.py", line 258, in get_note_by_id res = await self.post(uri, data) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\爬虫\1\MediaCrawler\media_platform\xhs\client.py", line 150, in post return await self.request( ^^^^^^^^^^^^^^^^^^^ File "C:\Users\chxii\AppData\Roaming\Python\Python312\site-packages\tenacity\asyncio_init_.py", line 189, in async_wrapped return await copy(fn, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\chxii\AppData\Roaming\Python\Python312\site-packages\tenacity\asyncio_init_.py", line 111, in call do = await self.iter(retry_state=retry_state) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\chxii\AppData\Roaming\Python\Python312\site-packages\tenacity\asyncio_init_.py", line 153, in iter result = await action(retry_state) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\chxii\AppData\Roaming\Python\Python312\site-packages\tenacity_utils.py", line 99, in inner return call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\chxii\AppData\Roaming\Python\Python312\site-packages\tenacity_init_.py", line 421, in exc_check raise retry_exc from fut.exception() tenacity.RetryError: RetryError[<Future at 0x20d66486ff0 state=finished raised Exception>]

📷 错误截图

Ellie-xi avatar Jul 20 '25 15:07 Ellie-xi

我也出现了相同的问题，请问有找到解决方案吗

onbn avatar Jul 21 '25 06:07 onbn

我也遇到了

NolynLin avatar Jul 21 '25 07:07 NolynLin

看错误日志是账号出现验证码了

我刚测试了一下，能跑没有触发验证码

尝试删除browser_data目录之后再试

NanmiCoder avatar Jul 21 '25 11:07 NanmiCoder

看错误日志是账号出现验证码了

我刚测试了一下，能跑没有触发验证码

尝试删除browser_data目录之后再试

我之前尝试过删除了browser_data，还是会触发

onbn avatar Jul 21 '25 11:07 onbn

看错误日志是账号出现验证码了 我刚测试了一下，能跑没有触发验证码 尝试删除browser_data目录之后再试

我之前尝试过删除了browser_data，还是会触发

我测试了账号没有触发风控，所以目前不知道又增强了什么风控策略，暂无解决方案，后续看看情况

NanmiCoder avatar Jul 21 '25 11:07 NanmiCoder

拉取最新代码

拿了一个小号经过测试，获取帖子详情会触发验证码，这个时候打开浏览器随便点击一个帖子链接会发现跳安全验证，用手机设备扫了之后就可以了。

应该是近期增强风控后的行为

NanmiCoder avatar Jul 21 '25 13:07 NanmiCoder

感谢解答，我拉取最新代码以后按照上面的方法成功了。想请问一下这个爬取帖子的数量有最大限制吗

拉取最新代码

拿了一个小号经过测试，获取帖子详情会触发验证码，这个时候打开浏览器随便点击一个帖子链接会发现跳安全验证，用手机设备扫了之后就可以了。

应该是近期增强风控后的行为

Ellie-xi avatar Jul 21 '25 15:07 Ellie-xi

还有一个问题就是爬取出来的csv文件的日期是错误的，请问一下要怎么修改代码

Image

Ellie-xi avatar Jul 22 '25 01:07 Ellie-xi

拉取最新代码

拿了一个小号经过测试，获取帖子详情会触发验证码，这个时候打开浏览器随便点击一个帖子链接会发现跳安全验证，用手机设备扫了之后就可以了。

应该是近期增强风控后的行为

感谢解答

onbn avatar Jul 22 '25 14:07 onbn