banditore icon indicating copy to clipboard operation
banditore copied to clipboard

Implement Retry after middleware for markdown endpoint

Open j0k3r opened this issue 8 years ago • 0 comments

The markdown endpoint is subject to the abuse rate limit. Which means after fetching new releases for new repository with a lot of releases, the markdown endpoint is called for each release (in CheckNewVersionCommand) and can generate abuse rate limit. The release won't be saved.

Instead of jumping to next tag, we should implement the Retry-After header returned from Github for that particular case.

Here is the exception thrown:

object(Github\Exception\RuntimeException)#25031 (7) {
  ["message":protected]=>
  string(96) "You have triggered an abuse detection mechanism. Please wait a few minutes before you try again."
  ["string":"Exception":private]=>
  string(0) ""
  ["code":protected]=>
  int(403)
  ["file":protected]=>
  string(106) "/home/www/bandito.re/www/vendor/knplabs/github-api/lib/Github/HttpClient/Plugin/GithubExceptionThrower.php"
  ["line":protected]=>
  int(87)
  ["trace":"Exception":private]=>
  array(19) {
    [0]=>
    array(6) {
      ["file"]=>
      string(85) "/home/www/bandito.re/www/vendor/php-http/httplug/src/Promise/HttpFulfilledPromise.php"
      ["line"]=>
      int(34)
      ["function"]=>
      string(34) "Github\HttpClient\Plugin\{closure}"
      ["class"]=>
      string(47) "Github\HttpClient\Plugin\GithubExceptionThrower"
      ["type"]=>
      string(2) "->"
      ["args"]=>
      array(1) {
        [0]=>
        object(GuzzleHttp\Psr7\Response)#25058 (6) {
          ["reasonPhrase":"GuzzleHttp\Psr7\Response":private]=>
          string(9) "Forbidden"
          ["statusCode":"GuzzleHttp\Psr7\Response":private]=>
          int(403)
          ["headers":"GuzzleHttp\Psr7\Response":private]=>
          array(16) {
            ["Server"]=>
            array(1) {
              [0]=>
              string(10) "GitHub.com"
            }
            ["Date"]=>
            array(1) {
              [0]=>
              string(29) "Sun, 19 Feb 2017 12:44:42 GMT"
            }
            ["Content-Type"]=>
            array(1) {
              [0]=>
              string(31) "application/json; charset=utf-8"
            }
            ["Transfer-Encoding"]=>
            array(1) {
              [0]=>
              string(7) "chunked"
            }
            ["Status"]=>
            array(1) {
              [0]=>
              string(13) "403 Forbidden"
            }
            ["Retry-After"]=>
            array(1) {
              [0]=>
              string(2) "60"

The retry middleware from Guzzle 6 can handle that case.

  • https://addshore.com/2015/12/guzzle-6-retry-middleware/
  • https://github.com/guzzle/guzzle/blob/master/tests/RetryMiddlewareTest.php
  • https://github.com/guzzle/guzzle/blob/master/src/RetryMiddleware.php

We should add a custom class to build the Guzzle Client and inject this middleware.

Here is a start about services to inject custom Guzzle client into the Github Client API.

services:
    banditore.client.guzzle:
        class: GuzzleHttp\Client

    # guzzle adaptater to inject custom client into Github client API
    banditore.guzzle.adapter:
        class: Http\Adapter\Guzzle6\Client
        arguments:
            - "@banditore.client.guzzle"

    # github http builder to inject custom Guzzle client
    banditore.client.github.http_builder:
        class: Github\HttpClient\Builder
        arguments:
            - "@banditore.guzzle.adapter"

    # global Github application client
    banditore.client.github.application:
        class: Github\Client
        arguments:
            - "@banditore.client.github.http_builder"
        calls:
            - [ authenticate, [ "%github_client_id%", "%github_client_secret%", !php/const:Github\Client::AUTH_URL_CLIENT_ID ] ]

j0k3r avatar Feb 19 '17 13:02 j0k3r