Warn on redirects

[tooomm]

301 into 404 doesn't give me an error anymore.

This is the URL in question to give you an example: http://deckmaster.info/images/cards/BNG/-66703-hr.jpg I'm using the GitHub Action of lychee.

With lychee v0.10.0 I got an error on that URL. With v0.10.1 the same URL reports a success.

v0.10.0 / [email protected] (I used to extract the links from the xml with XMLStarlet first, and wrote them to a text file)

--no-progress --require-https -- extracted_urls_from_original_file.txt

v0.10.1 / [email protected]

--no-progress --require-https --cache --max-cache-age 1h -- original_file.xml

The URL and its status codes can be inspected e.g. here, or in more detail on this site.

Should redirects be listed as warnings or hints to make users aware and give them the chance to adapt to the source directly? When moved permanently, there is a chance that the original link will break at one point, too.

Note: https://httpstat.us/ is a cool page that can help with easily checking certain status codes.

[mre]

Not sure I understand. I think it works as expected for me?

❯❯❯ lychee --version                                                                                                                                                             ✘
lychee 0.10.1

❯❯❯ echo 'http://deckmaster.info/images/cards/BNG/-66703-hr.jpg' | lychee -                                                                                                     
Issues found in 1 input. Find details below.

[stdin]:
✗ [404] http://deckmaster.info/images/cards/BNG/-66703-hr.jpg | Network error: Not Found

🔍 1 Total ✅ 0 OK 🚫 1 Error (HTTP:1)

Testing it directly with HTTPS also works:

❯❯❯ echo 'https://deckmaster.info/images/cards/BNG/-66703-hr.jpg' | lychee -                                                                                                     ✘
Issues found in 1 input. Find details below.

[stdin]:
✗ [404] https://deckmaster.info/images/cards/BNG/-66703-hr.jpg | Network error: Not Found

🔍 1 Total ✅ 0 OK 🚫 1 Error (HTTP:1)

So maybe I'm missing something? Could it be --cache?

[tooomm]

I think there was some problem on my side.

However, maybe this is more of a request then, that 301 should not be silently followed and report the status of the redirected to, new target. Those will most likely be successful (not in the example above) and nobody realizes that there was a redirect in place which can lead to issues in the future.

Should redirects be listed as warnings or hints to make users aware and give them the chance to adapt to the source directly? When moved permanently, there is a chance that the original link will break at one point, too.

Targets that moved might have temporary redirects set up. Those might be removed at one point though. But when that happens, there is no way to look up the new location that the 301 was reporting before. lychee would only report a 404.

Is it possible to highlight such redirects to give maintainers the chance to fix those ahead of time and don't run into such issues? Even reporting them as errors is probably fine and better than following the 301 and say everything works?

[mre]

Even reporting them as errors is probably fine and better than following the 301 and say everything works?

Related: https://github.com/lycheeverse/lychee-action/issues/164 If we fix this we have a workaround for this case: setting --max-redirects to 0. In that case 301s would result in an error.

[mre]

https://github.com/lycheeverse/lychee/pull/987 fixes --max-redirects behavior.

For example

 echo 'https://httpstat.us/301' | lychee --max-redirects 0 -

fails now as expected (with a build from master, not with a released version yet). Same for your URL.

echo 'http://deckmaster.info/images/cards/BNG/-66703-hr.jpg' | lychee --max-redirects 0 -
Issues found in 1 input. Find details below.

[stdin]:
✗ [ERR] http://deckmaster.info/images/cards/BNG/-66703-hr.jpg | Failed: Too many redirects: error following redirect for url (http://deckmaster.info/images/cards/BNG/-66703-hr.jpg): too many redirects

🔍 1 Total ✅ 0 OK 🚫 1 Error (HTTP:1)

Can you double-check?

[mre]

@tooomm, I assume this is fixed now and the fix got released with lychee 0.12.0. If this is not the case, feel free to reopen.