luigi icon indicating copy to clipboard operation
luigi copied to clipboard

postgres.CopyToTable: Robustize resources cleanup

Open LinqLover opened this issue 4 years ago • 1 comments

This PR aims to make connections using the contrib.postgres module more robust. If an error occurs during copying the rows, psycopg2 objects previously could be left opened without a proper clean-up, causing further connection attempts to the database to fail as well (psycopg2.errors.ObjectInUse).

By ensuring proper use of try...catch blocks and context managers, issues of this kind should be fixed now. Also, this PR includes regression tests.

For more information about releasing psycopg2 objects, visit: https://www.psycopg.org/docs/usage.html#with-statement

LBNL, a more precise exception class is used in the default copy() implementation of CopyToTable (f66534c6ae018d0cfddb7d09aa50cd1fcf6d36d1).

Kind of replaces #2934, which I unfortunately never had managed to complete.

Preferably please squash when merging.

LinqLover avatar Apr 15 '21 10:04 LinqLover

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If closed, you may revisit when your time allows and reopen! Thank you for your contributions.

stale[bot] avatar Jan 09 '22 01:01 stale[bot]