抖音search起不来，弹窗二维码弹起来了好多，有什么排查思路吗

[chunpat]

版本

git提交版本 commit 5f5147a2365e9632d665af84daeff6c1d231f350 (HEAD -> main, origin/main, origin/HEAD) Author: Relakkes [email protected] Date: Mon Mar 18 22:55:40 2024 +0800

docs: 添加捐赠者名单

问题

抖音search起不来，弹窗二维码弹起来了好多，然后查看控制台输出，判断是参数js加密信息不完整之类的失败了，怀疑是登录失败，所以删了登录信息重新，还是一样。

实现过程

执行

python main.py --platform dy --lt qrcode --type search

调起浏览器，弹出图片很多，我扫了最上层一个

查看控制器

(venv) ➜  MediaCrawler git:(main) ✗ python main.py --platform dy --lt qrcode --type search
search
2024-04-05  13:34:06 MediaCrawler INFO [DouYinCrawler.search] Begin search douyin keywords 
2024-04-05  13:34:06 MediaCrawler INFO [DouYinCrawler.search] Current keyword: 男生如何打扮 
Traceback (most recent call last):
  File "/Volumes/work/www/zzhpeng/MediaCrawler/main.py", line 62, in <module>
    asyncio.get_event_loop().run_until_complete(main())
  File "/usr/local/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Volumes/work/www/zzhpeng/MediaCrawler/main.py", line 53, in main
    await crawler.start()
  File "/Volumes/work/www/zzhpeng/MediaCrawler/media_platform/douyin/core.py", line 80, in start
    await self.search()
  File "/Volumes/work/www/zzhpeng/MediaCrawler/media_platform/douyin/core.py", line 96, in search
    posts_res = await self.dy_client.search_info_by_keyword(keyword=keyword,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/work/www/zzhpeng/MediaCrawler/media_platform/douyin/client.py", line 132, in search_info_by_keyword
    return await self.get("/aweme/v1/web/general/search/single/", params, headers=headers)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/work/www/zzhpeng/MediaCrawler/media_platform/douyin/client.py", line 80, in get
    await self.__process_req_params(params, headers)
  File "/Volumes/work/www/zzhpeng/MediaCrawler/media_platform/douyin/client.py", line 64, in __process_req_params
    x_bogus = douyin_js_obj.call('sign', query, headers["User-Agent"])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/work/www/zzhpeng/MediaCrawler/venv/lib/python3.11/site-packages/execjs/_abstract_runtime_context.py", line 37, in call
    return self._call(name, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/work/www/zzhpeng/MediaCrawler/venv/lib/python3.11/site-packages/execjs/_external_runtime.py", line 92, in _call
    return self._eval("{identifier}.apply(this, {args})".format(identifier=identifier, args=args))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/work/www/zzhpeng/MediaCrawler/venv/lib/python3.11/site-packages/execjs/_external_runtime.py", line 78, in _eval
    return self.exec_(code)
           ^^^^^^^^^^^^^^^^
  File "/Volumes/work/www/zzhpeng/MediaCrawler/venv/lib/python3.11/site-packages/execjs/_abstract_runtime_context.py", line 18, in exec_
    return self._exec_(source)
           ^^^^^^^^^^^^^^^^^^^
  File "/Volumes/work/www/zzhpeng/MediaCrawler/venv/lib/python3.11/site-packages/execjs/_external_runtime.py", line 88, in _exec_
    return self._extract_result(output)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/work/www/zzhpeng/MediaCrawler/venv/lib/python3.11/site-packages/execjs/_external_runtime.py", line 167, in _extract_result
    raise ProgramError(value)
execjs._exceptions.ProgramError: TypeError: Cannot read property 'JS_MD5_NO_COMMON_JS' of null

查了下JS_MD5_NO_COMMON_JS这个提醒不就是在libs/douyin.js，是加密请求参数时候用到。