sled
sled copied to clipboard
API for checkpoint / snapshot / backup system
Use Case:
Rocks equivalent:
- https://github.com/facebook/rocksdb/wiki/Checkpoints
- https://docs.rs/rocksdb/0.15.0/rocksdb/checkpoint/struct.Checkpoint.html
The primary use case here is exactly as described in the Rocks use case. Full & incremental backups. In my specific use case, I am building a distributed data storage system which uses async-raft for consensus, which includes a protocol where snapshots are sent to new nodes to bring them up-to-speed. Good stuff.
Proposed Change:
The API I am envisioning is as follows.
-
sled::DB
gets a new methodcheckpoint(path: impl AsRef<Path>)
, which does the work of generating a new DB checkpoint which will be written to the givenpath
. - Given the nature of checkpoints, it seems logical that this method only be exposed on the
DB
type and not onTree
s (maintainers would definitely know better on this one). - Some discussion is merited around incremental checkpoints / backups and what patterns would be best for this.
Who Benefits From The Change(s)?
Anyone and everyone looking for full / incremental backups, and the various use cases which emerge from that capability.
Alternative Approaches
Instead of calling this new API method checkpoint
, we could call it backup
.
@thedodd Take a look at https://github.com/spacejam/sled/issues/587#issuecomment-580276392 as well as the rest of that issue. Can we combine all of the ideas into a single new feature request?