linkinator
linkinator copied to clipboard
linkinator and curl disagree on link status
linkinator reports a status of 0 for https://developer.android.com/reference/android/os/Build as shown here:
$ npx linkinator https://developer.android.com/reference/android/os/Build --verbosity DEBUG
🏊♂️ crawling https://developer.android.com/reference/android/os/Build
[0] https://developer.android.com/reference/android/os/Build
[0] https://developer.android.com/reference/android/os/Build
[
{
"message": "maximum redirect reached at: https://developer.android.com/reference/android/os/Build",
"type": "max-redirect",
"config": {
"method": "GET",
"url": "https://developer.android.com/reference/android/os/Build",
"headers": {
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
},
"responseType": "stream",
"timeout": null
}
},
null
]
ERROR: Detected 1 broken links. Scanned 1 links in 3.258 seconds.
But if I use, say, curl
or visit the link manually and look at the network traffic, all I see are 200 status codes. For example:
$ curl -o - -I https://developer.android.com/reference/android/os/Build
HTTP/2 200
last-modified: Thu, 10 Feb 2022 17:54:54 GMT
content-type: text/html; charset=utf-8
...
Why the difference?
I'm seeing something similar with a microsoft.com link, and a few other sites
$ npx linkinator https://www.microsoft.com/en-us/microsoft-365/excel --verbosity DEBUG
🏊♂️ crawling https://www.microsoft.com/en-us/microsoft-365/excel
[0] https://www.microsoft.com/en-us/microsoft-365/excel
[0] https://www.microsoft.com/en-us/microsoft-365/excel
[
{
"message": "network timeout at: https://www.microsoft.com/en-us/microsoft-365/excel",
"type": "request-timeout",
"config": {
"method": "GET",
"url": "https://www.microsoft.com/en-us/microsoft-365/excel",
"headers": {
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
},
"responseType": "stream",
"timeout": 20000
}
},
null
]
ERROR: Detected 1 broken links. Scanned 1 links in 20.01 seconds.
vs.
curl -o - -I https://www.microsoft.com/en-us/microsoft-365/excel
HTTP/2 200
accept-ranges: bytes
content-length: 1020
content-type: text/html
etag: "6082151bd56ea922e1357f5896a90d0a:1425454794"
last-modified: Wed, 04 Mar 2015 07:39:54 GMT
server: AkamaiNetStorage
date: Mon, 16 Jan 2023 17:22:24 GMT
Ditto on this. I'm trying to run linkinator against a localdev site which i can curl without issue, but linkinator gives me a 0 error code.