Exclude items based on xattr

[m3nu]

Borg already processes extended attributes (borg.xattr) and has the --exclude-nodump to exclude items based on file flags. (set via chflags nodump $FILE)

On macOS (com.apple.metadata:com_apple_backup_excludeItem) and potentially Linux desktops (user.xdg.robots.backup=false ), extended attributes are used to exclude files from backups. Example from macOS:

$ xattr ~/Downloads
com.apple.macl
com.apple.metadata:com_apple_backup_excludeItem

So my suggestion would be to add a flag that accepts an OS-specific xattr as argument to exclude any item with this xattr from the backup. E.g. --exclude-xattr "com.apple.metadata:com_apple_backup_excludeItem"

What do you guys think of such a feature? I imagine that the implementation effort wouldn't be terribly high and I'm happy to prepare a PR, when pointed to the right places.

Related:

• User question for Vorta GUI: https://github.com/borgbase/vorta/issues/399
• Suggestion in Restic: https://github.com/restic/restic/issues/1242
• Sources for exclusions for macOS Time Machine: https://apple.stackexchange.com/a/25833/284053 (xattr are just one)

[ThomasWaldmann]

From stackexchange:

sudo mdfind "com_apple_backup_excludeItem = 'com.apple.backupd'"

So it seems not just to be some xattr key, but there is also a value.

What does com.apple.backupd mean? Would borg check for that key and value?

[m3nu]

From what I saw, the value is always the same. But to be future-proof, it could compare both. Depends on the performance difference. Currently Borg reads this from an excluded file:

In [0]: xattr.get_all(path, follow_symlinks=False)
Out[3]: ...
'com.apple.metadata:com_apple_backup_excludeItem': b'bplist00_\x10\x11com.apple.backupd\x08\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x1c'

[enkore]

>>> import plistlib
>>> plistlib.loads(b'bplist00_\x10\x11com.apple.backupd\x08\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x1c')
'com.apple.backupd'

Is there a spec for this? It seems kinda weird to encode just a string using plists to store it as just a string... :sweat_smile:

[ThomasWaldmann]

If an external tool can feed paths into borg based on their xattrs, this will be doable with borg 1.2.

[nicolas17]

If an external tool can feed paths into borg based on their xattrs, this will be doable with borg 1.2.

That would require scanning the filesystem and reading xattrs twice, once to generate the path list, and again by borg when making the backup (surely it reads the xattrs to back them up, right?).

[ThomasWaldmann]

Yeah, it would read (some) xattrs twice then.

[Artoria2e5]

Is there a spec for this?

As is the case with anything Apple, not officially. Wikipedia refers to some third-party sources.

It seems kinda weird to encode just a string using plists to store it as just a string... 😅

Since you are using Python's plistlib, I should probably mention that it neglects the plain old OpenStep plist format, which allows for encoding the string simply as either com.apple.backupd or "com.apple.backupd". Apple's own programs don't generate this format, but they do accept it. Lazy people who rely on one-liner scripts may end up creating this format.

(I do have a plistlib-like library for this, but it's not in any good shape. Just match the two exact strings.)

[ThomasWaldmann]

Merged #8895 to fix this.