cml-send-github-check does not work outside GH CI

[DavidGOrtega]

Solving #538 I have realised that the command does not work outside the CI.

Logs

(node:545) ExperimentalWarning: The fs.promises API is experimental
{ HttpError: You must authenticate via a GitHub App.
    at response.text.then.message (/opt/atlassian/pipelines/agent/build/node_modules/@octokit/request/dist-node/index.js:66:23)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  name: 'HttpError',
  status: 403,
  headers:
   { 'access-control-allow-origin': '*',
     'access-control-expose-headers':
      'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset',
     connection: 'close',
     'content-encoding': 'gzip',
     'content-security-policy': 'default-src \'none\'',
     'content-type': 'application/json; charset=utf-8',
     date: 'Tue, 06 Jul 2021 07:59:37 GMT',
     'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
     server: 'GitHub.com',
     'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
     'transfer-encoding': 'chunked',
     vary: 'Accept-Encoding, Accept, X-Requested-With',
     'x-accepted-oauth-scopes': '',
     'x-content-type-options': 'nosniff',
     'x-frame-options': 'deny',
     'x-github-media-type': 'github.v3; param=antiope-preview; format=json',
     'x-github-request-id': 'B970:0788:907789:A091D5:60E40D68',
     'x-oauth-scopes':
      'admin:enterprise, admin:gpg_key, admin:org, admin:org_hook, admin:public_key, admin:repo_hook, delete:packages, delete_repo, gist, notifications, repo, user, workflow, write:discussion, write:packages',
     'x-ratelimit-limit': '5000',
     'x-ratelimit-remaining': '4999',
     'x-ratelimit-reset': '1625561977',
     'x-ratelimit-resource': 'core',
     'x-ratelimit-used': '1',
     'x-xss-protection': '0' },
  request:
   { method: 'POST',
     url: 'https://api.github.com/repos/DavidGOrtega/cml/check-runs',
     headers:
      { accept: 'application/vnd.github.antiope-preview+json',
        'user-agent': 'octokit-core.js/3.1.2 Node.js/10.15.3 (linux; x64)',
        authorization: 'token [REDACTED]',
        'content-type': 'application/json; charset=utf-8' },
     body:
      '{"head_sha":"ae7864bf80b5badd572d05f55f8e403e67c5a7e8","started_at":"2021-07-06T07:59:36.895Z","completed_at":"2021-07-06T07:59:36.895Z","conclusion":"neutral","status":"in_progress","name":"CML Report","output":{"title":"CML Report","summary":"# My first CML report\\n"}}',
     request: { agent: [Agent], hook: [Function: bound bound register] } },
  documentation_url:
   'https://docs.github.com/rest/reference/checks#create-a-check-run' }

It happens that it needs to be authenticated as an app as stated here

[0x2b3bfa0]

Of course it doesn't: it only works with application tokens and ephemeral GitHub Actions tokens.

What can we do apart from documenting this behavior?

[DavidGOrtega]

What can we do apart from documenting this behavior?

Please read the stackoverflow link. It seems that its doable requiring the user to provide the JWT.

[0x2b3bfa0]

Please read the stackoverflow link. It seems that its doable requiring the user to provide the JWT.

Provided that we (or users) already have a GitHub App... unless I'm wrong. 🤔 Anyhow, if you manage to make it work with personal access tokens, users will probably be very happy.

[dacbd]

With this not being a very popular feature or very advertised on our end I think we can close this / open two new issues to create an implementation for gitlab and bb when we want to revisit this.