sdk icon indicating copy to clipboard operation
sdk copied to clipboard

feat: Build a default merge upsert implementation for SQL targets

Open aaronsteers opened this issue 3 years ago • 0 comments
trafficstars

As an alternative or complement to:

  • #1127

This would provide a default experience for SQL target developers which would handle merge upsert behavior in one or both of the following expressions:

Scenario 1: DB supports MERGE UPSERT (SQLStream.allow_merge_upsert = True)

If the backend does support upsert, the implementation can use MERGE UPSERT to load data from a temp table to the target table.

Scenario 2: DB doesn't support upsert (SQLStream.allow_merge_upsert = False)

If the backend does not support upsert, then the default implementation can either insert rows to the target table directly (not using a temp table) or else take a two-step process of (1) inserting new rows and (2) update existing roles.

aaronsteers avatar Oct 29 '22 05:10 aaronsteers