cloud-sql-python-connector icon indicating copy to clipboard operation
cloud-sql-python-connector copied to clipboard

Add pooled_connection option

Open baarden opened this issue 2 years ago • 1 comments

feat: option for useful cert refresh behavior for pooled connections

When a pool of connections is being maintained by SQLAlchemy, there is no need for the connections to do their own maintenance, since SQLAlchemy already closes unhealthy connections and opens new connections as needed. And because pooled connections do not get closed actively, the current refresh behavior can result in too many connections being held open, and surges of refreshes every 55 minutes if multiple pools were opened at the same time.

Similarly, the current behavior of immediately retrying a failed refresh can lead to a thundering herd of refreshes if the Cloud SQL API requests-per-minute quota has been exceeded. In a managed pool it will be safer to wait 60 seconds for the next quota period.

Separately, the connector is throwing asyncio errors (below), which are corrected by adding an explicit close() call to the __client_session after use. This action is moved to a separate method to avoid flake8 complexity complaints.

ERROR:asyncio:Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x10f1fd850>
ERROR:asyncio:Unclosed connector

Changes:

  • Adds a pooled_connection option to Connector and Instance which modifies the behavior of certification refreshes. When true, timed refreshes are disabled. Also, failed refreshes are retried after 60 seconds.
  • A new _close_session method is added to Instance to prevent unclosed ClientSession errors.
  • Tests also require a GOOGLE_APPLICATION_CREDENTIALS envvar value, so that suggestion is added to CONTRIBUTING.md.

Testing: Passing tests continue to pass with these changes.

baarden avatar Jul 31 '22 20:07 baarden

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

google-cla[bot] avatar Jul 31 '22 20:07 google-cla[bot]

Closing due to inactivity on #415

jackwotherspoon avatar Nov 28 '22 15:11 jackwotherspoon