replicator

replicator copied to clipboard

In scenarios where the target schema does not define tables of the same names used in an incoming CDC feed (e.g. if using a `dispatch` function in the userscript that...

bobvawter

Test Renaming tables crdb --> postgres

2

BramGruneir

It should be straightforward to have a standard performance benchmark command that will characterize the performance of a user's staging and target databases. This command should exercise the `stage` and...

bobvawter

Avoid marshalling in logical loop drivers

1

sravotto

Add histogram to track apply mutation batch size

2

We have apply_duration_seconds_bucket to track apply performance, however we don't track the size of mutation batches we are upserting/deleting.

sravotto

While these steps are available in the Postgres documentation, it would be useful to explicitly mention them: - wal_level should be set to "logical" - During bulk migration, "SELECT pg_export_snapshot()"...

sravotto

Suport backfilling pglogical connections

2

For simple use cases, we should support at least a simple way of copying the table data from the source db to the target cluster. If we use a `pg_export_snapshot()`,...

bobvawter

Think about bi-directional sync

3

Discussion issue for thinking about how two clusters could be kept in eventual-sync: * Source/destination cluster tracking column to avoid looping cdc-sink'ed mutations * MVCC timestamp comparison to drop too-old...

bobvawter

Playbook: Use a CRDB backup to bootstrap the target cluster

1

We need a playbook that describes how to use a CRDB backup to initially populate the target database, and then how to create a changefeed that starts from the backup's...

bobvawter

Support core changefeed as a logical loop

1

We should support consuming a CRDB core changefeed, to support a "pull"-based model if the source cluster is not able to create outgoing network connections to a cdc-sink deployment. This...

bobvawter