dead links ahoy

[pnathan]

https://www.site24x7.com/link-checker.html

The internet rots under us, alas. We can but repair.

http://www.psg.com/~dlamkins/sl/cover.html should, probably, be amended to http://successful-lisp.blogspot.com/p/httpsdrive.html

http://www.aiai.ed.ac.uk/~jeff/lisp/cl-pitfalls is now gone; it looks like ~jeff is gone.

[odanoburu]

http://www.aiai.ed.ac.uk/~jeff/lisp/cl-pitfalls is now gone; it looks like ~jeff is gone.

wayback machine to the rescue! https://web.archive.org/web/20171007090745/http://www.aiai.ed.ac.uk/~jeff/lisp/cl-pitfalls

[markboston34]

is this issue still looking for someone to help?

[vindarel]

if you find an automated tool for that, that would be very nice ! (there must be many out there)

[alx-a]

Hope it's ok i'm upping here instead of creating a new issue. Just pulled a request on an updated link to the CL standard, the cvberry website appears to have become a link farm. More details in the request. It appears link rot is a serious concern, that particular web squatting link is full of aggressive JS.

Not sure if an automated link-checking tool would be possible, as the squatter in my example just sent a redirect as opposed to "not found". I'm going to slowly review other external links in the cookbook. The more people do this, the better.

[alx-a]

So I ran markdown-link-check, and found some more 404s. markdown-link-check can be integrated into GitHub workflows

• Discuss if implementing markdown-link-check is worth the trouble? Let me know if setting up a separate issue for automated checking would be preferred.

• Of course the broken Web of today means that many servers don't provide the courtesy of a 404 anymore, so manual checking and re-checking still applies. Insert into contributing? i.e. Every update you submit, please check all the links manually.

• Since the links in question often need discussion, doing small individual PRs, hope it's ok, otherwise please let me now.

[vindarel]

This anti-404 Github action would help :) Do we need it on every push though? Is there a solution to run it periodically? (I'm only thinking it would be wasted energy, I am optimizing prematurely…) Does a [ci skip] text in a message commit skips the CI with Github Actions? (that's a Gitlab CI thing)

edit: for a schedule:

schedule:
  # Run everyday at 9:00 AM (See https://pubs.opengroup.org/onlinepubs/9699919799/utilities/crontab.html#tag_20_25_07)
  - cron: "0 9 * * *"

[alx-a]

@vindarel I suppose checking on every push is not necessary. I do check anyway. If I'm going to do a PR, I check the links on the cookbook page I'm working on. But requesting / defaulting link checking will most certainly discourage drive-by PRs that are valuable for typos etc.

I'm going to experiment with changedetection.io and see if it's useful. There is also page monitor. Any feedback or ideas on existing solutions to tracking web-pages that could be easily adapted into our workflow are very much appreciated.

Otherwise I'll just try to do a crond download of key reference pages and probably monitor for more than 1% change via git diff.

To re-repeat, core issue outside of 404s is updates to content that don't send up a 404, as seen from my recent PR, so some kind of content monitoring change solution is in order. I'll ping an update when I have tested over some time and shall also continue semi-manual checks on existing pages.

As you commented, vindarel, for my documentation draft, I will cut down the external links and increase the recipes. They rot less :)