borgbackup
Borg create paths silently override patterns
Have you checked borgbackup docs, FAQ, and open Github issues?
Yes.
Is this a BUG / ISSUE report or a QUESTION?
BUG
System information. For client/server mode post info for both machines.
Your borg version (borg -V).
1.2.1. Also repros in Borg 2.0 beta 1.
Operating system (distribution) and version.
Manjaro Linux 21.3.7.
Hardware / network configuration, and filesystems used.
Lenovo X1 Carbon, local repository only, ext4.
How much data is handled by borg?
On the order of megabytes in these tests.
Full borg commandline that lead to the problem (leave away excludes and passwords)
borg create --pattern="R /root/tmp" --debug --show-rc 1.2.borg::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} /root/tmp/subdir
Describe the problem you're observing.
When I borg list the resulting archive, only /root/tmp/subdir is backed up. I'd expect /root/tmp (as listed in the pattern) to get backed up as well. If I run the same command omitting /root/tmp/subdir entirely, then /root/tmp is backed up. Similarly, if I move /root/tmp/subdir off the command-line and into a pattern as a root, then both roots get backed up.
Other variants that result in the same "buggy" behavior:
- Use two paths where one isn't inside the other.
- Use
--patterns-frominstead of--pattern. - Use Borg 2.0 beta 1 (with a Borg 2 repository) instead of 1.2.1, adjusting the command-line flags accordingly.
Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.
Yes, it reproduces every time, running the command above on an existing repository.
Include any warning/errors/backtraces from the system logs
# borg create --pattern="R /root/tmp" --debug --show-rc 1.2.borg::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} /root/tmp/subdir
using builtin fallback logging configuration
33 self tests completed in 0.07 seconds
Verified integrity of /root/tmp/1.2.borg/index.85
TAM-verified manifest
security: read previous location '/root/tmp/1.2.borg'
security: read manifest timestamp '2022-08-25T18:01:22.929429'
security: determined newest manifest timestamp as 2022-08-25T18:01:22.929429
security: repository checks ok, allowing access
Creating archive at "1.2.borg::flux-2022-08-25T11:01:32.675698"
Verified integrity of /root/.cache/borg/80d73d55f45bd377b76de84d24a208b3c20c644e3871e52ea8afefd22a583408/chunks
Reading files cache ...
Verified integrity of /root/.cache/borg/80d73d55f45bd377b76de84d24a208b3c20c644e3871e52ea8afefd22a583408/files
security: read previous location '/root/tmp/1.2.borg'
security: read manifest timestamp '2022-08-25T18:01:22.929429'
security: determined newest manifest timestamp as 2022-08-25T18:01:22.929429
security: repository checks ok, allowing access
Processing files ...
Cleaned up 0 uncommitted segment files (== everything after segment 85).
Verified integrity of /root/tmp/1.2.borg/hints.85
check_free_space: required bytes 545430238, free bytes 4761632768
security: saving state for 80d73d55f45bd377b76de84d24a208b3c20c644e3871e52ea8afefd22a583408 to /root/.config/borg/security/80d73d55f45bd377b76de84d24a208b3c20c644e3871e52ea8afefd22a583408
security: current location /root/tmp/1.2.borg
security: key type 3
security: manifest timestamp 2022-08-25T18:01:32.830588
terminating with success status, rc 0
# borg list 1.2.borg::flux-2022-08-25T11:01:32.675698
drwxr-xr-x root root 0 Thu, 2022-08-25 11:01:30 root/tmp/subdir
-rw-r--r-- root root 0 Thu, 2022-08-25 11:01:30 root/tmp/subdir/file.txt
Guess this should just "add up" / use both.
Patterns and excludes IIRC behave that way, one can give them in pattern/exclude files as well as as cli params and they will all be used.
Added print(args.paths) in do_create:
$ borg2 create --repo /tmp/b2 --pattern="R 1" --pattern="R 2" archive 3 4 --pattern="R 5" --pattern="R 6"
['3', '4', '5', '6']
So, it's not completely broken (5 and 6 are in there), but order matters and as soon as it processes the first positional PATH arg (3), it seems to reset the list.
Maybe something to add there?:
https://borgbackup.readthedocs.io/en/stable/man_intro.html#positional-arguments-and-options-order-matters
Except it looks like the current behavior, in which positional patterns arguments need to come first to prevent silent overrides, is kind of the opposite of the "good and preferred" argument order example on that documentation page, in which positional arguments come last?
maybe we are lucky:
https://docs.python.org/3/library/argparse.html#action see "extend" there (new since python 3.8).
the default is "store" and that seems to start from scratch (ignoring any existing list) when processing the positional args.