pattern matching, docs vs. reality

[ThomasWaldmann]

While dealing with #7631, we noted that this patterns file does not work as I expected:

P sh
R /Users/tw/w/borg
+ Users/tw/w/borg/src
+ Users/tw/w/borg/src/borg
+ Users/tw/w/borg/src/borg/helpers
+ Users/tw/w/borg/src/borg/helpers/**
- **

The idea was to include the parent dir fs items (only the 3 directory items src, borg and helpers, not the files inside), then include the desired files (here: all inside helpers) and exclude everything else.

Issue: It included all below Users/tw/w/borg/src into the backup archive (not just the dir items and the files in helpers).

So it seems not to work as I thought it would (I personally do my backups the other way: include all, only exclude what I do not want).

Update:

This is already enough to get that behaviour:

P sh
R /Users/tw/w/borg
+ Users/tw/w/borg/src
- **

Reproduced with borg 1.2.3, 1.2.4 and 1.2-maint git branch.

[ThomasWaldmann]

Using pf: prefix only for the parent dirs makes this work as desired.

[ThomasWaldmann]

I had a look at borg.testsuite.patterns and now I think we need a rewrite of pattern docs and tests in a way that the docs and tests precisely reflect each other.

The input file paths in the tests should better reflect real use cases as encountered on typical linux filesystems and backup scenarios.

Also, I guess there need to be some simplifications and less magic behaviour, KISS (keep it simple and stupid).

As this might change matching behaviour, guess this is a borg2-only change.

[ThomasWaldmann]

Depending on the changes done in this refactoring, this might be a breaking change.

Like: the users need to verify whether their patterns still work as intended and fix them if needed.

[c0d3z3r0]

Maybe also add appropriate test cases to the testsuite

[ThomasWaldmann]

@c0d3z3r0 https://github.com/borgbackup/borg/issues/7634#issuecomment-1584890185