feat(data-warehouse): Added an append only replication mode to dwh

[Gilbert09]

Problem

• Sometimes we wanna set up syncs to not merge data into an existing dataset, but instead to append the new data only
• For example, Stripe, where we can only filter on the created timestamp, and so we'll never actually merge data together, and thus we're wasting a lot of resources trying to merge in when really we could just be appending the data instead

Changes

• Adds a new "append only" replication method to imports
  • This still takes a field to filter the source data on, such as a created_at field
• This will only append incoming data to the table and not merge anything

Next:

• I'd like to show/hide replication methods in the UI dependent on the source (e.g. Stripe should really be only "append only" or full refresh)
  • I'm keen to switch all existing incremental stripe tables over to this new method
• Maybe some more logging around this would be handy for future debugging too

Did you write or update any docs for this change?

• Some docs need to be written up before this gets merged about the new replication method

How did you test this code?

• Tested a bunch locally and added an end to end test

[github-actions[bot]]

Size Change: 0 B

Total Size: 2.57 MB

ℹ️ View Unchanged

Filename	Size
frontend/dist/toolbar.js	2.57 MB

_{compressed-size-action}

[posthog-bot]

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

[posthog-bot]

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

[!CAUTION]

Detected flapping snapshots

These snapshots have auto-updated more than once since the last human commit:

• scenes-app-insights-funnels--funnel-top-to-bottom-edit--light.png (chromium, shard 2)

The flippy-flappies are deadly and must be fixed ASAP. They're productivity killers. Run pnpm storybook locally and make the fix now. (Often, the cause is ResizeObserver being used instead of the better CSS container queries.)

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

[posthog-bot]

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

[posthog-bot]

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

[!CAUTION]

Detected flapping snapshots

These snapshots have auto-updated more than once since the last human commit:

• scenes-app-insights-funnels--funnel-top-to-bottom-breakdown--light.png (chromium, shard 2)

The flippy-flappies are deadly and must be fixed ASAP. They're productivity killers. Run pnpm storybook locally and make the fix now. (Often, the cause is ResizeObserver being used instead of the better CSS container queries.)

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

[posthog-bot]

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

• chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

[posthog-bot]

📸 UI snapshots have been updated

3 snapshot changes in total. 0 added, 3 modified, 0 deleted:

[!CAUTION]

Detected flapping snapshots

These snapshots have auto-updated more than once since the last human commit:

• scenes-app-insights-funnels--funnel-top-to-bottom-breakdown-edit--light.png (chromium, shard 2)

The flippy-flappies are deadly and must be fixed ASAP. They're productivity killers. Run pnpm storybook locally and make the fix now. (Often, the cause is ResizeObserver being used instead of the better CSS container queries.)

• chromium: 0 added, 3 modified, 0 deleted (diff for shard 1, diff for shard 2)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

[posthog-bot]

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

[!CAUTION]

Detected flapping snapshots

These snapshots have auto-updated more than once since the last human commit:

• scenes-other-onboarding--onboarding-billing--dark.png (chromium, shard 1)
• scenes-other-onboarding--onboarding-billing--light.png (chromium, shard 1)

The flippy-flappies are deadly and must be fixed ASAP. They're productivity killers. Run pnpm storybook locally and make the fix now. (Often, the cause is ResizeObserver being used instead of the better CSS container queries.)

• chromium: 0 added, 2 modified, 0 deleted (diff for shard 1)
• webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

[github-actions[bot]]

Migration SQL Changes

Hey 👋, we've detected some migrations on this PR. Here's the SQL output for each migration, make sure they make sense:

posthog/migrations/0772_alter_externaldataschema_sync_type.py

BEGIN;
--
-- Alter field sync_type on externaldataschema
--
-- (no-op)
COMMIT;