Action is incompatible with `pull_request` triggers from forks/dependabot

[dbjorge]

Currently, if you trigger the action via a forked repo (or a native dependabot PR), the GITHUB_TOKEN provided to the workflow does not have write permissions for checks, so you get an error from the action when it receives a 403 response during its check creation/update requests.

The full error signature looks like this

[Exception] === VError: An error occurred while scanning website page undefined: Resource not accessible by integration
    at Logger.trackExceptionAny (/home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:70931:29)
    at Scanner.<anonymous> (/home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71922:29)
    at Generator.throw (<anonymous>)
    at rejected (/home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71853:65)
    at processTicksAndRejections (internal/process/task_queues.js:95:5) {
  jse_shortmsg: 'An error occurred while scanning website page undefined',
  jse_cause: RequestError [HttpError]: Resource not accessible by integration
      at /home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:5508:23
      at processTicksAndRejections (internal/process/task_queues.js:95:5) {
    status: 403,
    headers: {
      'access-control-allow-origin': '*',
      'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset',
      connection: 'close',
      'content-encoding': 'gzip',
      'content-security-policy': "default-src 'none'",
      'content-type': 'application/json; charset=utf-8',
      date: 'Tue, 25 May 2021 21:04:51 GMT',
      'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
      server: 'GitHub.com',
      'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
      'transfer-encoding': 'chunked',
      vary: 'Accept-Encoding, Accept, X-Requested-With',
      'x-content-type-options': 'nosniff',
      'x-frame-options': 'deny',
      'x-github-media-type': 'github.v3; format=json',
      'x-github-request-id': '0400:43D4:3336731:636CA48:60AD6673',
      'x-ratelimit-limit': '15000',
      'x-ratelimit-remaining': '14994',
      'x-ratelimit-reset': '1621979696',
      'x-ratelimit-resource': 'core',
      'x-ratelimit-used': '6',
      'x-xss-protection': '0'
    },
    request: {
      method: 'POST',
      url: 'https://api.github.com/repos/microsoft/accessibility-insights-action/check-runs',
      headers: {
        accept: 'application/vnd.github.v3+json',
        'user-agent': 'octokit-rest.js/18.5.3 octokit-core.js/3.4.0 Node.js/14.17.0 (linux; x64)',
        authorization: 'token [REDACTED]',
        'content-type': 'application/json; charset=utf-8'
      },
      body: '{"name":"Accessibility Checks","status":"in_progress","head_sha":"d396568bd3c95e53e11a3f79d34487240174ca31"}',
      request: { hook: [Function: bound bound register] }
    },
    documentation_url: 'https://docs.github.com/rest/reference/checks#create-a-check-run'
  },
  jse_info: {}
}
[Trace][info] === Updating check run with status as failed
[Trace][info] === Accessibility scanning of URL undefined completed
Exception thrown in action:  TypeError: Cannot read property 'id' of undefined
    at CheckRunCreator.<anonymous> (/home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71360:50)
    at Generator.next (<anonymous>)
    at /home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71293:71
    at new Promise (<anonymous>)
    at __webpack_modules__../src/progress-reporter/check-run/check-run-creator.ts.__awaiter (/home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71289:12)
    at /home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71356:37
    at CheckRunCreator.<anonymous> (/home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71444:49)
    at Generator.next (<anonymous>)
    at /home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71423:71
    at new Promise (<anonymous>)
    at __webpack_modules__../src/progress-reporter/progress-reporter.ts.__awaiter (/home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71419:12)
    at CheckRunCreator.invoke (/home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71443:16)
    at CheckRunCreator.<anonymous> (/home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71356:24)
    at Generator.next (<anonymous>)
    at /home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71293:71
    at new Promise (<anonymous>)
    at __webpack_modules__../src/progress-reporter/check-run/check-run-creator.ts.__awaiter (/home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71289:12)
    at CheckRunCreator.failRun (/home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71352:16)
    at /home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71242:41
    at AllProgressReporter.<anonymous> (/home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71249:23)
    at Generator.next (<anonymous>)
    at /home/runner/work/accessibility-insights-action/accessibility-insights-action/dist/index.js:71215:71
Error: Process completed with exit code 1.

Uploading a build artifact is allowed with the read-only token permissions, so the workaround I've seen other similar actions (eg, dorny/test-reporter's suggestion) use is to have the user include the action twice; once in a on: pull_request workflow that runs the scan and uploads the results as a build artifact, and a second time in a separate on: workflow_run workflow that triggers when the first workflow completes, runs using a more permissive GITHUB_TOKEN, and adds a check based on the build artifact from the on: pull_request results.

Ideally, we'd:

• Implement support for the two-phase action run
• Document how a user would set that up in usage.md (similar to dorny/test-reporter's "Recommended setup for public repositories" docs
• Detect the 403 and output a more actionable error message that points the user to that documentation instead of spewing out 100 lines of stack trace

[msftbot[bot]]

This issue requires additional investigation by the Accessibility Insights team. When the issue is ready to be triaged again, we will update the issue with the investigation result and add "status: ready for triage". Thank you for contributing to Accessibility Insights!

[chrisreddington]

+1 - I've just encountered this on CloudWithChris/cloudwithchris.com#106

[asksep]

The Accessibility Insights team will be adding a new scenario to release validation process and will then mark this bug as closed.

[dbjorge]

Updated the release validation template to clarify that verifying the action works in a PR scenario should be tested from a fork.