Check for broken links

[kinow]

@Fienne found a broken link in the user guide. That reminded me of two tools that can be used for finding issues like this. The second can also be used to prevent, by hooking it up in our CI system.

1. W3 checklink: https://validator.w3.org/checklink

It checks anchors, internal, and external links. You need to provide the parameters that can be saved in a cookie locally. It takes a few seconds/minutes, but gives you a pretty detailed report to go through and fix any broken links.

2. Sphinx linkcheck

Not sure if it can be configured to inspect external links. It checks anchors and internal links. Comes with Sphinx. It is a builder like the html builder. Here's an example run, grep'ing for the token "broken":

$ sphinx-build -b linkcheck src _build/ | grep broken
(introduction/prerequisites: line   58) broken    https://github.com/common-workflow-language/cwltool#install - Anchor 'install' not found
(topics/specifying-software-requirements: line   25) broken    https://github.com/common-workflow-language/cwltool#leveraging-softwarerequirements-beta - Anchor 'leveraging-softwarerequirements-beta' not found
(topics/file-formats: line   45) broken    https://github.com/common-workflow-language/user_guide/raw/main/_includes/cwl/file-formats/file-formats.bam - 404 Client Error: Not Found for url: https://github.com/common-workflow-language/user_guide/raw/main/_includes/cwl/file-formats/file-formats.bam
(introduction/prerequisites: line  173) broken    https://github.com/rabix/benten#install-vs-code-extension - Anchor 'install-vs-code-extension' not found
(topics/custom-types: line   24) broken    https://raw.githubusercontent.com/common-workflow-language/user_guide/main/_includes/cwl/custom-types/rich_sparse_otu_table.biom - 404 Client Error: Not Found for url: https://raw.githubusercontent.com/common-workflow-language/user_guide/main/_includes/cwl/custom-types/rich_sparse_otu_table.biom
(introduction/prerequisites: line  176) broken    https://www.commonwl.org/#Editors_and_viewers - Anchor 'Editors_and_viewers' not found

I did not check if the links are indeed broken. But we used it in the Cylc project (I believe they still use it, and for external links? → https://github.com/cylc/cylc-doc/blob/5a5aaa111c6756fdf7c4ab9b2f8ec6cabd37ae6e/src/conf.py#L136-L140).

@swzCuroverse maybe a good one for Outreachy? A little harder to work on this as it requires some researching into how Sphinx works (although it should be nerdily fun! :nerd_face: ).

-Bruno

[Fienne]

@Fienne found a broken link in the user guide. That reminded me of two tools that can be used for finding issues like this. The second can also be used to prevent, by hooking it up in our CI system.

1. W3 checklink: https://validator.w3.org/checklink

It checks anchors, internal, and external links. You need to provide the parameters that can be saved in a cookie locally. It takes a few seconds/minutes, but gives you a pretty detailed report to go through and fix any broken links.

2. Sphinx linkcheck

Not sure if it can be configured to inspect external links. It checks anchors and internal links. Comes with Sphinx. It is a builder like the html builder. Here's an example run, grep'ing for the token "broken":

$ sphinx-build -b linkcheck src _build/ | grep broken
(introduction/prerequisites: line   58) broken    https://github.com/common-workflow-language/cwltool#install - Anchor 'install' not found
(topics/specifying-software-requirements: line   25) broken    https://github.com/common-workflow-language/cwltool#leveraging-softwarerequirements-beta - Anchor 'leveraging-softwarerequirements-beta' not found
(topics/file-formats: line   45) broken    https://github.com/common-workflow-language/user_guide/raw/main/_includes/cwl/file-formats/file-formats.bam - 404 Client Error: Not Found for url: https://github.com/common-workflow-language/user_guide/raw/main/_includes/cwl/file-formats/file-formats.bam
(introduction/prerequisites: line  173) broken    https://github.com/rabix/benten#install-vs-code-extension - Anchor 'install-vs-code-extension' not found
(topics/custom-types: line   24) broken    https://raw.githubusercontent.com/common-workflow-language/user_guide/main/_includes/cwl/custom-types/rich_sparse_otu_table.biom - 404 Client Error: Not Found for url: https://raw.githubusercontent.com/common-workflow-language/user_guide/main/_includes/cwl/custom-types/rich_sparse_otu_table.biom
(introduction/prerequisites: line  176) broken    https://www.commonwl.org/#Editors_and_viewers - Anchor 'Editors_and_viewers' not found

I did not check if the links are indeed broken. But we used it in the Cylc project (I believe they still use it, and for external links? → https://github.com/cylc/cylc-doc/blob/5a5aaa111c6756fdf7c4ab9b2f8ec6cabd37ae6e/src/conf.py#L136-L140).

@swzCuroverse maybe a good one for Outreachy? A little harder to work on this as it requires some researching into how Sphinx works (although it should be nerdily fun! nerd_face ).

-Bruno

I'm glad I found the broken link 😅. These tools look very interesting and I'll definitely be using them. Thanks @kinow !!

[swzCuroverse]

@kinow should we close this out - or would you like to integrate the first suggestion into the build process so we check every time?

[kinow]

I won't have time for now, but maybe somebody else could be interested in implementing a check for that. Maybe it could be fine-tuned to verify links for certain domains (e.g. commonwl.org), or just scan every link, etc.

[swzCuroverse]

@Mackenzie-OO7 and @Fienne would you be interested in adding a check for this?

[Mackenzie-OO7]

@Mackenzie-OO7 and @Fienne would you be interested in adding a check for this?

Yes, I would.