attic
attic copied to clipboard
jborg
Parallel backups from different hosts crash attic
Attic: debian package 0.13-1 The target is one shared archive for all hosts. Backup is via local lan and ssh, except one host which uses local direct backup.
I want to run backups from several hosts via ssh to one backup host. Running those backups at the same time will crash attic:
Traceback (most recent call last):
File "/usr/bin/attic", line 3, in
OR
Traceback (most recent call last):
File "/usr/bin/attic", line 3, in
I need to run them one after each other. This should be fixed.
@gullevek can you still reproduce this with a current attic version?
A problem is that the traceback does not contain enough information about the AttributeError that happened on the remote side.
A problem is that the traceback does not contain enough information about the AttributeError that happened on the remote side.
slightly offtopic, but can you tell me how to get a traceback from the remote side (in general)? Does the attic serve call log its messages to some particular place by default?
I just installed attic for the backup of all my systems (around 10 computers). all run debian jessie and thus attic in debian version 0.13-1. the backups are simply done by remote ssh to the same host with the same attic archive. All backups start at the same time with a command like this on every host via cronjob:
attic create --do-not-cross-mountpoints [email protected]:/backups/attic::goofy.blabla.lan_root /
but i get a lot of errors like shown in the above conversations.
are there some issues with running attic in parallel? do i have to add some locking? any ideas?
actually i am a little disappointed that such a basic issue is open. attic looked so professional and promising to me, and now it completely fails right in the first night. the resulting archive is corrupted. at least it should be documented that manual selfmade locking is mandatory?
I just checked the system by running the backup scripts one after another manually, and all is fine. it only fails when run in parallel.
next week i will probably add manual locking to the backup scripts.
@erikhydro have a look at borgbackup (which is a fork of attic), a lot of issues are fixed there. iirc it is available from the backports repo. there is also a single-file binary.
but "parallel" backups to same repository will be serialized (as it should be with attic also) and will also require re-synchronization of the chunks index, so it depends whether one or separate repos are the better idea.
https://attic-backup.org/quickstart.html#remote-repositories here i can read, that i can use a remote repository by mounting and then using local attic, but then i must make sure that no two instances are running in parallel. That i have read and understood. But i concluded, that when using ssh and not a local mounted repository, then i could run in parallel.
can you confirm, that attic does NOT serialize and thus will corrupt the archive? so it is not somehow my error?
if yes i will commit another bug onto the debian bugtracker. there is already a grave one: https://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=jessie;package=attic because i have the strong feeling that administrators should really know that they can NOT use attic by ssh to backup their server farm! (if not taking extra care to not run parallel, but that's so dangerous to forget. there really should be a report.)
If you don't have working posix locks (due to using a filesystem that does not correctly support them), you are out of luck with attic. I assumed you have client/server attic after seeing the commandline you posted.
That is the main reason why borg uses mkdir-based locking. That basically works everywhere, even on rather crappy filesystems. It has one disadvantage though: the lock does not automatically go away when the process gets killed rather violently (kill -9, power failure, ...).