Retry logic when rate-limited

[darraghmckay]

This PR adds optional logic that will retry every request that returns in a 429 response. A 429 means you have been rate-limited according to these usage-limits

This is off by default and is only enabled by running

doc.setRetryOptions(retries, retryDelay)

It felt kinda weird setting it on a doc level rather than a library level but I wasn't sure what was best. Definitely open to suggestions there.

Fixes: https://github.com/theoephraim/node-google-spreadsheet/issues/418

[wingedfox]

This is quite a must-have option, without the library-level support for the back-offs it's almost impossible to develop anything.

[TaylorFacen]

Checking in on the status of this PR. Built-in retry logic would be very helpful.

[darraghmckay]

It doesn't seem like there's been any activity on this project in quite a while unfortunately I can't merge this without approval from the repo admin

[agladysh]

NB: Google Sheets API lists rate limits and suggests a specific backoff algorithm to follow:

https://developers.google.com/sheets/api/limits

[charlesgardyn]

While this is not merged, you can use axios-retry:

import { AxiosInstance } from 'axios'
import axiosRetry from 'axios-retry'
import { JWT } from 'google-auth-library'
import { GoogleSpreadsheet } from 'google-spreadsheet'
import _ from 'lodash'

function retry(axiosInstance: AxiosInstance) {
  axiosRetry(axiosInstance, {
    retries: 7,
    retryDelay: retryCount => {
      const randomNumberMS = _.random(1000, 8000)
      return Math.min(4 ** retryCount + randomNumberMS, maximumBackoff)
    },
    retryCondition: error => error.response.status === 429,
  })
}

const auth = new JWT({
    email: client_email,
    key: private_key,
    scopes: ['https://www.googleapis.com/auth/spreadsheets'],
  })
const doc = new GoogleSpreadsheet(sheetId, auth)

retry(doc.sheetsApi)
retry(doc.driveApi)

[ruscon]

While this is not merged, you can use this retry approach for downloadAsTSV

  // node@^15 
  import { setTimeout } from 'node:timers/promises'
  import type { GoogleSpreadsheetWorksheet } from 'google-spreadsheet'

  async downloadAsTSV(sheet: GoogleSpreadsheetWorksheet): Promise<ArrayBuffer> {
    try {
      return await sheet.downloadAsTSV()
    } catch (error) {
      // axios@^1.0.0
      if (error instanceof AxiosError && error.response?.status === 429) {
        await setTimeout(10000)
        console.warn(`retry downloadAsTSV for "${sheet.title}"`)

        return downloadAsTSV(sheet)
      }

      throw error
    }
  }