抓取archive.today

[duty-machine]

使用node-fetch抓取archive today的页面总是被要求输入验证码，不知道是什么原因。即使我使用和chrome同样的headers，以下的header是直接从chrome里复制出来的：

let res = await fetch(url, {
      "headers": {
        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "accept-language": "en",
        "cache-control": "no-cache",
        "pragma": "no-cache",
        "sec-fetch-dest": "document",
        "sec-fetch-mode": "navigate",
        "sec-fetch-site": "none",
        "sec-fetch-user": "?1",
        "upgrade-insecure-requests": "1",
        "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"
      },
      "referrerPolicy": "strict-origin-when-cross-origin",
      "body": null,
      "method": "GET",
      "mode": "cors"
})

初步认为是node-fetch夹带了其他能暴露自己的header，但是不太清楚怎么查

[duty-machine]

本地用curl可以拉到，但是github action里用curl也拉不到