litefs icon indicating copy to clipboard operation
litefs copied to clipboard

Object Store Mediated Replication

Open pg-wtatum opened this issue 1 month ago • 0 comments

Related to #18, it would be extremely powerful to be able to replicate data to read-replicas outside the datacenter infrastructure which could not securely or performantly access the HTTP endpoints of the litefs master directly. My proposed use-case if offline first web and mobile clients that can tolerate somewhat higher latency in their replication than the current method used for primary to read-replica, but still want an efficient and repeatable method for replicating database state from a primary.

If #18 is implemented then it feels like a very powerful added capability this would offer is for detached clients to use S3 as a method to get "in sync" with the primary (minus a time delay) without needing to manage any connectivity with the primary. The major additional effort to support this would be the ability to trigger a "restore" operation of the backup independent of the rest of the litefs plumbing (and most likely from an application sdk rather than via filesystem operations). Ideally this would only require the client to have S3 coordinates for the various objects holding the backup but not any other litefs config.

pg-wtatum avatar Jun 25 '24 21:06 pg-wtatum