anytype-heart icon indicating copy to clipboard operation
anytype-heart copied to clipboard

badger backup system

Open requilence opened this issue 1 year ago • 1 comments

for spacestore

  • run incremental backup every minute
  • every 1000 backups(empty skipped) do the full backup and remove all others after
  • do the os.Sync before renaming backups from *.tmp
  • in case it was not able to open badger on start recover from backup
  • only in case backup restore was successful the old corrupted dir got renamed and the restored one take it place
  • because we never rewrite most of the keys' values in spacestore(only object heads), incremental backup works effective and doesn't consume a lot of space

for localstore

  • in case of error on open just reinit the db without backup
  • corrupted repo left for debugging

Also Need to switch to this PR's branch https://github.com/dgraph-io/badger/pull/2033 The reason is that in the most cases the panic happens in the goroutine and it not possible to catch it in from the badger.Open caller

I think backuper may be used only on WIndows. Let's see how often this corruptions happen on UNIX

requilence avatar Jan 08 '24 21:01 requilence

New Coverage 43.7% of statements
Patch Coverage 47.1% of changed statements (105/223)


Coverage provided by https://github.com/seriousben/go-patch-cover-action

github-actions[bot] avatar Jan 11 '24 10:01 github-actions[bot]