github icon indicating copy to clipboard operation
github copied to clipboard

refactor: to use octokit throttling plugin

Open moltar opened this issue 3 years ago • 7 comments

This is still WIP, but I need help.

  1. Some of tests are failing and I cannot explain.
  2. A set of tests in get-client is failing, but I am not sure if they still make sense, given that we are switching to the plugin, which handles all of the logic. See below. Thoughts?
  3. Tests started to run much slower after adding the plugins, which makes me think that maybe they are bypassing nock and hitting live endpoints? Or maybe it is just the throttling/retry that does that.
✖ get-client › Use the global throttler for all endpoints Rejected promise returned by test
✖ get-client › Use the same throttler for endpoints in the same rate limit group Rejected promise returned by test
✖ get-client › Use different throttler for read and write endpoints Rejected promise returned by test
✖ get-client › Use the same throttler when retrying Rejected promise returned by test

Closes #377

moltar avatar Jun 01 '21 01:06 moltar

you can remove all tests for throttling as we don't need to test internals of the throttle plugin, it's well tested already

gr2m avatar Jun 01 '21 17:06 gr2m

Removed the throttling tests.

Still a few failures left.

One peculiar one is in test/verify.test.js.

Looks like something with nock, but can't yet figure it out.

> nyc ava -v "test/verify.test.js" "-T" "100s"


  ✖ Throw SemanticReleaseError for invalid token Rejected promise returned by test
  ─

  Throw SemanticReleaseError for invalid token

  Rejected promise returned by test. Reason:

  HttpError (RequestError) {
    headers: {},
    request: {
      headers: {
        accept: 'application/vnd.github.v3+json',
        authorization: 'token [REDACTED]',
        'user-agent': 'octokit-rest.js/18.5.3 octokit-core.js/3.4.0 Node.js/14.15.5 (darwin; x64)',
      },
      method: 'GET',
      request: {
        agent: undefined,
        hook: Function bound bound register {},
        retries: 3,
        retryAfter: 16,
        retryCount: 3,
      },
      url: 'https://api.github.com/repos/test_user/test_repo',
    },
    status: 500,
    message: `request to https://api.github.com/repos/test_user/test_repo failed, reason: Nock: No match for request {␊
      "method": "GET",␊
      "url": "https://api.github.com/repos/test_user/test_repo",␊
      "headers": {␊
        "accept": [␊
          "application/vnd.github.v3+json"␊
        ],␊
        "user-agent": [␊
          "octokit-rest.js/18.5.3 octokit-core.js/3.4.0 Node.js/14.15.5 (darwin; x64)"␊
        ],␊
        "authorization": [␊
          "token github_token"␊
        ],␊
        "accept-encoding": [␊
          "gzip,deflate"␊
        ],␊
        "connection": [␊
          "close"␊
        ]␊
      }␊
    }`,
  }

  › }
  › node_modules/@octokit/request/dist-src/fetch-wrapper.js:91:15
  › Job.doExecute (node_modules/bottleneck/light.js:405:18)

Putting this on pause for a bit. If anyone wants to jump in, feel free.

moltar avatar Jun 02 '21 05:06 moltar

Still a few failures left.

One peculiar one is in test/verify.test.js.

Looks like something with nock, but can't yet figure it out.

I'll have a look

gr2m avatar Jun 02 '21 22:06 gr2m

I've pushed a few changes, I resolved two failing tests but gotta go now. Can you check if you can resolve the rest?

gr2m avatar Jun 03 '21 01:06 gr2m

I've pushed a few changes, I resolved two failing tests but gotta go now. Can you check if you can resolve the rest?

Awesome! Good teamwork 😁

Any idea why nock debugging is not working?

DEBUG=nock:* npm t -- test/fail.test.js

I get no debug output at all from nock.

moltar avatar Jun 03 '21 03:06 moltar

Ugh, never mind, figured it out.

nock seems to deviate from the node convetion of using : as a separator.

The right way is to use a dot . -- DEBUG=nock.*.

moltar avatar Jun 03 '21 03:06 moltar

Apologies if this is perceived as spam, but just wanted to drop a note of thanks for starting this work, and hopefully it can merge at some point - I just had a busted release from github's secondary rate limits on a project :sob: - the release itself went out, but the issue / PR comments didn't go, so it wasn't tragic, but it wasn't as beautiful as the releases normally are :-). Cheers

mikehardy avatar Jan 17 '22 06:01 mikehardy

Done via https://github.com/semantic-release/github/releases/tag/v8.0.8

gr2m avatar May 27 '23 22:05 gr2m