litestream icon indicating copy to clipboard operation
litestream copied to clipboard

Allow setting replica interval offsets

Open hifi opened this issue 2 years ago • 2 comments

If using S3 replicas the extra LIST call on startup may be expensive in scale if a lot of Litestreams are started in quick succession.

Allowing configuring the snapshot offset externally allows the caller to spread around the snapshots without relying on access to the remote replica making restarts no-op.

Same thing applies to retention checks that they can easily stack too close.

hifi avatar Nov 15 '23 08:11 hifi

I'll modify this a little as we need an offset for retention check as well.

hifi avatar Nov 16 '23 12:11 hifi

Expanded to cover all intervals. We use both snapshot and retention check offsets to prevent storming a bucket with requests around the same time.

hifi avatar Nov 17 '23 12:11 hifi