litestream icon indicating copy to clipboard operation
litestream copied to clipboard

A RESTART checkpoint during a snapshot transfer locks the database for writing until the snapshot has finished

Open t-voet opened this issue 2 years ago • 2 comments

Litestream v0.3.5 and a 25 GB database on ZFS with 64k page_size. A SFTP snapshot takes hours, during which the WAL hits 10k pages (640 MB). An obvious solution could be skipping RESTART checkpoints while a snapshot transfer is in progress.

t-voet avatar Oct 03 '21 16:10 t-voet

Thanks, @t-voet. I'll take a look at optimizing SFTP and skipping the RESTART checkpoints during transfer for v0.4.0.

benbjohnson avatar Oct 06 '21 22:10 benbjohnson

You're the best. If you send me your public key, I can provide a high-speed (2.5 Gbps) SFTP account for testing.

t-voet avatar Oct 07 '21 09:10 t-voet

As I believe #477 fixed this I'll close. Thanks.

hifi avatar Dec 27 '23 15:12 hifi