sdk icon indicating copy to clipboard operation
sdk copied to clipboard

feat: Add rollback support for overwrite load method

Open pnadolny13 opened this issue 2 years ago • 1 comments

Originally driven by https://github.com/meltano/sdk/issues/1783 which was closed by https://github.com/meltano/sdk/pull/1893 but doesnt have safe backup/rollback support for the overwrite method.

The current status of this is described in https://github.com/meltano/sdk/pull/1893#issuecomment-1710693275 and a draft PR is in https://github.com/meltano/sdk/pull/1941.

The overwrite load method currently drops the table before attempting to load so if the sync fails then the user is left with an empty table (or no table if recreate fails). This issue is to create a workflow to:

  • alter existing table to back it up instead of dropping it
  • attempt to recreate the table and load the new data
  • if success, delete the back up table
  • if failed, delete the new table and rename the backup table to the original name

pnadolny13 avatar Sep 13 '23 13:09 pnadolny13

This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.

stale[bot] avatar Sep 12 '24 16:09 stale[bot]