electric icon indicating copy to clipboard operation
electric copied to clipboard

Store replication state outside of Postgres to detect repeat transactions

Open alco opened this issue 6 months ago • 0 comments

https://github.com/electric-sql/electric/pull/1511 added a test for idempotent processing of repeat transactions in our current shape storage implementation. That, however, does not cover the case where another storage implementation is introduced which does not have the same property.

Ideally we would store sufficient replication state outside of Postgres such that we could tell whether a given transaction arriving from the replication stream has already been processed before and drop it right there, without forwarding it to the storage driver.

This is related to https://github.com/electric-sql/electric/issues/1506.

alco avatar Aug 12 '24 21:08 alco