community-committee icon indicating copy to clipboard operation
community-committee copied to clipboard

test: add nightly markdown checking

Open bnb opened this issue 4 years ago • 8 comments

Adds nightly markdown link checking. Helps prevent broken links that are left alone forever.

bnb avatar Aug 17 '20 22:08 bnb

currently failing because we're getting throttled by GitHub since we have so many GitHub links in this repository 😅

bnb avatar Aug 17 '20 22:08 bnb

This is currently completely blocked unless I can either ignore a path (/meetings) or batch requests. I've made issues (https://github.com/tcort/markdown-link-check/issues/102 and https://github.com/tcort/markdown-link-check/issues/101, respectively) for these in the correct repository. Until they're addressed, I don't think we can merge this. I'm totally open to alternative Actions that allow us to do this, this was just the best one I've found.

bnb avatar Aug 17 '20 23:08 bnb

I think we can do both using the config-file option, it takes a JSON file to configure the underlying markdown-link-check npm package: https://github.com/tcort/markdown-link-check#config-file-format.

{
	"ignorePatterns": [
		{
			"pattern": "^http://example.net"
		}
	],
	"replacementPatterns": [
		{
			"pattern": "^.attachments",
			"replacement": "file://some/conventional/folder/.attachments"
		}
	],
	"httpHeaders": [
		{
			"urls": [
				"https://example.com"
			],
			"headers": {
				"Authorization": "Basic Zm9vOmJhcg==",
				"Foo": "Bar"
			}
		}
	]
}

The file probably would need to be generated on each run. Something like (not tested, this could be wildly wrong):

  - run: |
    cat <<EOF > config.json
    {
      "ignorePatterns": [
      {
        "pattern": "^http://github.com/${{ github.repository }}/tree/.*/meetings/.*"
      }
      ],
      "httpHeaders": [
        {
          "urls": [
            "https://github.com"
          ],
          "headers": {
            "Authorization": "Bearer ${{ secrets.GITHUB_TOKEN }}",
          }
        }
      ]
    }
    EOF

mmarchini avatar Aug 17 '20 23:08 mmarchini

@mmarchini could you expand on why it'd need to be generated on each run? The HTTP Header part may be enough itself, I wasn't able to find any docs saying that was enabled/allowed on GitHub's side 🤔

bnb avatar Aug 17 '20 23:08 bnb

Because we receive a new token on each run, and since the file is a JSON we can't set the authentication headers programatically on the config file. If markdown-link-check accepted JS files as well for config (like webpack and eslint do), we wouldn't need to generate on each run.

mmarchini avatar Aug 17 '20 23:08 mmarchini

ah, that makes sense 🤦🏻

bnb avatar Aug 17 '20 23:08 bnb

Added that cat command - I need to read up more on that usage of cat since it doesn't seem to have worked precisely how it should have (the build should have failed, but it succeeded 🤔 ).

I could have also put it in the wrong spot.

bnb avatar Aug 17 '20 23:08 bnb

theoretically the PR linked below should have fixed the issue, I'll see if I can update the PR soon.

https://github.com/tcort/markdown-link-check/pull/106

bnb avatar Oct 06 '20 18:10 bnb

I've unarchived this repo so I can close all PRs and issues before re-archiving.

Trott avatar Apr 22 '23 16:04 Trott