dice icon indicating copy to clipboard operation
dice copied to clipboard

Point in time snapshots

Open gsarmaonline opened this issue 9 months ago • 1 comments

The PR implements snapshotting a single store into the disk using the copy-on-write approach.

Check the README.md in the PR for more details on the design and implementation.

The PR doesn't implement the strategy of multi shard snapshots yet. It will be implemented in a subsequent PR.

Test results

Snapshot time for 1 million keys:

  • No Reads and no writes is 190ms.
  • Reads and no writes is 230ms.
  • Reads and with writes during the snapshots is 800ms.

The reads and writes mentioned above are during the snapshotting process on the actual store object.

Few improvements that can be done:

  • Use protobuf instead of gob
  • Use buffered writes instead of chunk writes
  • Improve the locking mechanism of the SnapshotMap

gsarmaonline avatar Feb 09 '25 02:02 gsarmaonline

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Mar 20 '25 11:03 CLAassistant