litestream icon indicating copy to clipboard operation
litestream copied to clipboard
verified publisher icon benbjohnson

LTX Support

Open benbjohnson opened this issue 3 years ago • 2 comments

The LiteFS project uses the LTX file format for storing change sets and there a lot of improvements that can be made to Litestream if it used the same format:

  • Compaction support
  • WAL offset & salt maintained in LTX header
  • Compression support can be moved to LTX
  • Encryption support (planned)

benbjohnson avatar Jan 28 '23 18:01 benbjohnson

As the streaming replication support that was canned made changes to the file structure, will LTX support also keep the breaking change or allow restoring from legacy (0.3.x) WAL files and structure at least for one minor version (0.4.x) to make migration seamless and safe?

hifi avatar Jan 28 '23 19:01 hifi

@hifi I'm on the fence about backward compatibility. The new version will take a snapshot right away when it starts up so it'll be seamless and safe from that perspective. In theory, it could handle pulling in historic backups but that'd also be a fair bit of code to maintain that support as well.

benbjohnson avatar Jan 28 '23 22:01 benbjohnson