Integrity error: 300k files 100GB of data.

[brodul]

Hi,

First of all I would like to thank you for this project. I am testing attic for backing up a lot of media (pictures). I am getting integrity errors after the backup is done, when I run attic check.

It could be that the integrity errors are a result of HW error or something. But it can also be an attic bug.

Do you have any idea how to debug this or if there is a way that I can help.

[bjornfor]

Another (new) user here, looking to switch to attic from rsync-based backup. I got integrity error after hitting a full disk during attic create.

I freed up some disk space and tried to fix the error with attic check --repair, and got this:

$ time sudo attic check --repair /media/3tb/backups/backup.attic/
attic: Warning: 'check --repair' is an experimental feature that might result
in data loss.

Type "Yes I am sure" if you understand this and want to continue.

Do you want to continue? Yes I am sure
Starting repository check...
Error reading segment 119001
attempting to recover /media/3tb/backups/backup.attic/data/11/119001
Traceback (most recent call last):
  File "/nix/store/j8g5xshwdzl8i9rl9sp8d4mi4xdlvfhw-attic-0.14/bin/.attic-wrapped", line 4, in <module>
    main()
  File "/nix/store/j8g5xshwdzl8i9rl9sp8d4mi4xdlvfhw-attic-0.14/lib/python3.4/site-packages/attic/archiver.py", line 727, in main
    exit_code = archiver.run(sys.argv[1:])
  File "/nix/store/j8g5xshwdzl8i9rl9sp8d4mi4xdlvfhw-attic-0.14/lib/python3.4/site-packages/attic/archiver.py", line 717, in run
    return args.func(args)
  File "/nix/store/j8g5xshwdzl8i9rl9sp8d4mi4xdlvfhw-attic-0.14/lib/python3.4/site-packages/attic/archiver.py", line 80, in do_check
    if repository.check(repair=args.repair):
  File "/nix/store/j8g5xshwdzl8i9rl9sp8d4mi4xdlvfhw-attic-0.14/lib/python3.4/site-packages/attic/repository.py", line 312, in check
    self.compact_segments()
  File "/nix/store/j8g5xshwdzl8i9rl9sp8d4mi4xdlvfhw-attic-0.14/lib/python3.4/site-packages/attic/repository.py", line 194, in compact_segments
    assert segments[segment] == 0
AssertionError

real    141m44.567s
user    12m45.602s
sys 4m29.892s

I thought that AssertError was fatal, but apparently it recovered enough for me to be able to run attic create again (which now chugs along...). I'm trying to backup ~800 GB, so I'd prefer to recover rather than start over, even if this is currently "a test".

[brodul]

I also want to mention that I am doing remote backups.

Now I am trying rsycing first and then creating an archive. Will see how that turns out.

[ThomasWaldmann]

Could you reproduce the integrity error? If so, can you please specify attic version and msgpack version?

[ThomasWaldmann]

@brodul and @bjornfor - please individually state whether you run attic on an ARM cpu - see similar traceback in #309.

[bjornfor]

@ThomasWaldmann: No ARM hardware here. (I used x86_64, although my issue was full disk. Don't know about @brodul.)

[brodul]

@ThomasWaldmann: No ARM on my servers. I quit the job in which I used attic.

I have plans to create some scripts (probably fabric) and run some tests on AWS or sponsored hardware. Would be interesting to have some function tests (making backups with the CLI), maybe Travis can run them, but I am not sure about limitations.

[ThomasWaldmann]

@brodul some of attic's tests are already rather close to commandline invocations. Not sure what testing on AWS would give us compared to local and travis CI test runners.

[brodul]

@ThomasWaldmann never mind :)

I took a look at the limitations of Travis. So technically one can have a job that takes 50minutes. Which means I can make a repo install attic and do some stuff with larger test assets (larger, more files) (I can generate them on the fly) in a relative deterministic environment. Would this help?

[ThomasWaldmann]

@brodul if your goal is scalability testing, I doubt that travis would be good for that. So if you want to improve tests, maybe rather have a look at coverage. Travis is not that fast and I don't think they would be happy if we do GB- or TB-scale or even huge-amount-of-little-files backup tests. :D

[brodul]

@ThomasWaldmann :D Well that was my concern in the first place.

[ThomasWaldmann]

@jborg I could reproduce the issue seen in https://github.com/jborg/attic/issues/166#issuecomment-68614341 and fixed it like that:

https://github.com/ThomasWaldmann/borg/commit/98f3b8d937dea79e86b0c2f9c608a094d01325eb

If you could review and provide feedback, that would be great. :)