materialize icon indicating copy to clipboard operation
materialize copied to clipboard

storage/sources/postgres: incremental snapshots

Open petrosagg opened this issue 2 years ago • 0 comments

Debezium supports taking an incremental snapshot of the database which is great when the replicated database is very large and taking a read lock for an extended period of time is undesirable. Materialize should support this mode of snapshot too.

The method is described in this paper https://arxiv.org/pdf/2010.12597v1.pdf and requires having write access to a table that participates to the replication. This probably means that we can't add it as a default but further investigation is needed.

petrosagg avatar Mar 06 '23 12:03 petrosagg