borgbackup
Add follow symbolic link support
Borg backup currently saves symbolic links as they are, not following them. I propose adding an option to follow symbolic links as flag to borg create.
Usecase: I use anything-sync-daemon and profile-sync-daemon to store Desktop/chrome profile in tmpfs and speed up my chrome startup time and speed.
However they work by creating symbolic links in place of real folders, and I would like to backup my home dir without such issues.
Well, if one only has cases where following symlinks at backup time and restoring something different at restore time is fine, I guess there are no problems.
BUT: as soon as you have some symlinks that better stay symlinks at restore time and some other symlinks that you want followed (== restored as their target), you are in trouble.
Good observation. Perhaps something as follow-symblinks = {never, always, same-file-system, regex-matching} option would provide enough flexibility while keeping complexity down.
Two things come to mind
- You can use a bind-mount instead (this is truly transparent, unlike a symlink)
- Add the symlinked folders explicitly
This looks to me like a very small niche and making it work requires little effort (unless there is something wrong with my suggestion, please tell :)
I moved the --read-special symlink issue to #1215, it is a special case while this ticket is about the generic symlink following.
@d2dyno no, and I have quite some doubts this is even a desirable feature.
The idea of a backup is that you get back the same state after restore and that means "not following symlinks, archiving symlinks as symlinks, restoring symlinks as symlinks" and that is what borg does right now. You just need to make sure you backup all filesystems / backup roots you need for recovery.
Also, as soon as you start following symlinks, there could be some interesting effects with endless recursion or that you might end up with wanting to follow some but not other symlinks.
Hi, I understand it may be causing some headaches but I have the following situation/setup where e.g. the ~/Documents folder is actually a link to another directory on another partition of the drive. This allows to have various OS (of the multiboot setup) to share the same documents etc. Inside the documents directory/partition here are other links to other directories/partitions because I was running out of space on the main documents directory/partition. I'm not sure but how does back-in-time/rsync manage symlinks ? Maybe the same approach could be followed. If the user has a switch he can choose what suits him best. Thanks
Personally I agree with Thomas- I have multiple symlinks that need to be restored exactly as they are if my server crashed, otherwise my entire file structure would be broken on restore. I think this would be the same for most people running Linux production servers.
Might be useful for some people as an additional feature but the current core functionality re: symlinks should definitely stay as it currently is
As a temporary solution you could try using rclone mount -L and run borg on mounted folder. This is merely a suggestion, as, though it worked for me with restic, it produced weird errors on borg duo to locked files on my setup.
I would just like to revisit this topic if that's ok...?
I agree with the idea that the backup restore should place the system in the same state as before backup, but I think a switch to change default behaviour is still beneficial. My use case is that I have ~/etc/ containing symlinks to /etc/ for system files that I have modified that I want included in my homedir backup. I'm only really interested in backing up my homedir with minimal system config changes that have been made and tracked in ~/etc/.
To get this behaviour I have to use tar with the follow symlinks option.
-h, --dereference
Follow symlinks; archive and dump the files they point to.
My homedir otherwise doesn't contain symlinks.
@Hammit That sounds like a good use case, but it does not really require that borg feature.
Assuming you keep all your "user modified" configs in ~/my-etc-configs, you can just add that to the PATHs you back up (assuming that the home directory is not backed up in this backup run anyway).
@ThomasWaldmann I think @Hammit has symlinks going from ~/etc/ to /etc/, so backing up ~/etc/ wouldn't back up the contents. But the solution, then, is to reverse the direction of the symlinks, in my opinion. (I would also make sure that the files in ~/etc/ are owned by root are are not world writable, for security.)
@jdchristensen Or just add /etc to the PATHs (it's not much and compresses well). The point is always having the symlink source AND target in the backup archive.
Thanks @ThomasWaldmann for the comments and input. One disadvantage to what you suggest, which is to be explicit about specifying each file in the backup paths is that new symlinks don't get backed up automatically. With my suggestion, you could specify ~/etc/ as a backup path and all contents are automatically backed up without having to remember to add new symlinks to the backup. Does that make sense? I think that is kind of important (for me at least).
@jdchristensen Your suggestion of moving the original files to ~/etc/ and symlinking from /etc/ I hadn't thought of. It does sound like a workaround to the fact that Borg can't follow symlinks like tar can however. Is this purely an ideological standpoint to take or are there practical reasons to not follow symlinks as well?
@Hammit I see what you mean.
But (for the workaround) I didn't mean specifying each file's paths, but something like borg create repo::archive /etc ~/etc ....
That would back up a bit more than the minimum, but /etc isn't that big and compresses well.
About following symlinks: guess this has some potential for complications and rabbit holes, so this would need a clear and detailed idea about how it should work.