MediaCrawler icon indicating copy to clipboard operation
MediaCrawler copied to clipboard

[问题] xhs搜索关键字及评论时,第一次是可以的,但是Ctrl+ C停止后,在次启动一直失败

Open fengyanWang opened this issue 4 months ago • 1 comments

⚠️ 提交前确认

  • [ ] 我已经仔细阅读了项目使用过程中的常见问题汇总
  • [ ] 我已经搜索并查看了已关闭的issues
  • [ ] 我确认这不是由于滑块验证码、Cookie过期、Cookie提取错误、平台风控等常见原因导致的问题

❓ 问题描述

🔍 使用场景

  • 目标平台: (如:小红书/抖音/微博等)
  • 使用功能: (如:关键词搜索/用户主页爬取等)

💻 环境信息

  • 操作系统:
  • Python版本:
  • 是否使用IP代理:
  • 是否使用VPN翻墙软件:
  • 目标平台(抖音/小红书/微博等):

📋 错误日志

在此粘贴完整的错误日志

2025-08-04 07:51:58 MediaCrawler INFO (core.py:270) - [get_note_detail_async_task] Begin get note detail, note_id: 61dff80400000000210371e5 2025-08-04 07:51:58 httpx INFO (_client.py:1740) - HTTP Request: POST https://edith.xiaohongshu.com/api/sns/web/v1/feed "HTTP/1.1 200 OK" 2025-08-04 07:51:59 httpx INFO (_client.py:1740) - HTTP Request: POST https://edith.xiaohongshu.com/api/sns/web/v1/feed "HTTP/1.1 200 OK" 2025-08-04 07:52:00 httpx INFO (_client.py:1740) - HTTP Request: POST https://edith.xiaohongshu.com/api/sns/web/v1/feed "HTTP/1.1 200 OK" 2025-08-04 07:52:00 httpx INFO (_client.py:1740) - HTTP Request: GET https://www.xiaohongshu.com/explore/61dff80400000000210371e5?xsec_token=ABiSy_KRXy9-R_kfSAPSB5IuFyT9dUNSJZudBCDYZzR5Y=&xsec_source=None "HTTP/1.1 302 Found" 2025-08-04 07:52:00 MediaCrawler INFO (core.py:270) - [get_note_detail_async_task] Begin get note detail, note_id: 65afc71b000000000c0072a0 Traceback (most recent call last): File "I:\project\pythonProject\dify_project\MediaCrawler\main.py", line 78, in asyncio.get_event_loop().run_until_complete(main()) File "C:\Users\20803747\AppData\Roaming\uv\python\cpython-3.9.23-windows-x86_64-none\lib\asyncio\base_events.py", line 647, in run_until_complete return future.result() File "I:\project\pythonProject\dify_project\MediaCrawler\main.py", line 65, in main await crawler.start() File "I:\project\pythonProject\dify_project\MediaCrawler\media_platform\xhs\core.py", line 104, in start await self.search() File "I:\project\pythonProject\dify_project\MediaCrawler\media_platform\xhs\core.py", line 157, in search note_details = await asyncio.gather(*task_list) File "I:\project\pythonProject\dify_project\MediaCrawler\media_platform\xhs\core.py", line 280, in get_note_detail_async_task raise Exception(f"[get_note_detail_async_task] Failed to get note detail, Id: {note_id}") Exception: [get_note_detail_async_task] Failed to get note detail, Id: 61dff80400000000210371e5

📷 错误截图

fengyanWang avatar Aug 03 '25 23:08 fengyanWang

触发小红书风控了,打开你程序运行时启动的浏览器,随便点进去一个小红书帖子,这时候应该会弹出来一个二维码,验证你是否真人登录,用手机扫一下就行,扫完一次后再爬就不会有这种风控问题了

2513502304 avatar Aug 04 '25 04:08 2513502304