Add support for retry with exponential backoff

[borland667]

Add retry with exponential backoff mechanism when the airtable API returns 429, 500, 502, 503, 504 http code responses

Added 4 new environment variables to enable retries, max retry count and connection pool sizes.

• API_CLIENT_ENABLE_RETRIES
• API_CLIENT_MAX_RETRIES
• API_CLIENT_POOL_CONNECTIONS
• API_CLIENT_MAX_POOL_SIZE

Feel free to suggest better names :)

[gtalarico]

I like this approach.

Some initial reactions:

• We will need to find a way to test this
• What's the lib API to customize retry behavior?
• IMHO default should be no retry at least for now but should be very easy to enable it to avoid changing lib behavior on existing code bases.

edit: removed note about mount()

[NicoHood]

Will a request ever timeout?

[gtalarico]

@NicoHood timeouts should be handled by the underlying requests lib.

Config parameters can be set with timeout kwarg: https://pyairtable.readthedocs.io/en/latest/api.html#api

The url lib retry object used in PR can be configured to control how it behaves when it encounters a connection or read timeout

https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html

[borland667]

I like this approach.

Some initial reactions:

* We will need to find a way to test this

* What's the lib API to customize retry behavior?

* IMHO default should be no retry at least for  now but should be very easy to enable it to avoid changing lib behavior on existing code bases.

edit: remove not about mount()

Thanks!!

A few things I've yet to do:

• Include a simple server with an url that returns random status codes and a final 200 OK so we can use it to test the retry logic.
• Add environment variables for controlling the retry behavior, at least: DEFAULT_BACKOFF_FACTOR, DEFAULT_MAX_RETRIES, DEFAULT_POOL_CONNECTIONS and DEFAULT_MAX_POOL_SIZE.
• Add a new config parameter to enable Retry or just enable it if DEFAULT_MAX_RETRIES > 0

Also, the other approach I thought of is to add the Retry object on every call that's done. However, we miss some of the connection pool functionality that the session provide.

We currently have this working on our connector to airtable and we've just extended the Base class and replaced the session with the AutoRetry one and it has been working fine. Our use case is a job that pulls all tables from airtable into our local mysql database.

[gtalarico]

@borland667

re: env vars: IMHO the env config works well for frameworks and applications but it's not ideal for libraries. library users can easily load env values and pass values in instead.

testability: spent more time than I am proud of but found a way to test this.

api design: proposed in PR.

I hope this is ok w/ you, but I forked your PR and made the changes mentioned above: let me know if you have any thoughts: https://github.com/gtalarico/pyairtable/compare/main...add-retry-exp-backoff

[borland667]

Awesome @gtalarico !! Thanks for that!! That's why I love open source :) Will have a look at the PR today!