Draft: borgbackup: server backup location

[fredericmoulins]

trafficstars

WIP attempt to see if it would be possible to move the backup folders and repos of the users created by the backup-server playbook from the backup repo of the server.

Currently, the layout uses:

• a backup name;
• a location: local with dir or remote with ssh, usb, cifs, s3fs;
• a mount point: /mnt/backup/<name> for usb, cifs, s3fs;
• a repository path: the location for dir and ssh, the mount point for usb, cifs, s3fs.

A borg repository is initialized at the repository path for the server backups. Folders for user backups are created under the repository path for storing user created borg repos. I assume this is to avoid having the borg backup on the server processing already existing borg repositories (a lot of unnecessary hash computations).

# ls -1F <repository path>
config
data/
<uid1>/
<uid2>/
hints.145
index.145
integrity.145
nonce
README

To avoid adding directories inside the repo directory controlled by borg, the idea would be to separate the mount point / location from the repository path, and use a layout such as:

# ls -1F <mount point | location>
@server/
<uid1>/
<uid2>/

The @server directory being the borg repository for the server backup. The @ is there to avoid naming conflicts as it cannot be used in a UID.

The 'server' name is arbitrary, it could as well be 'homebox'. The assumption is that these directories will be stored under a (remote) directory already identifying the server. Naming it '{{ network.domain }}' might be redundant.

The first two commit are unifying the locations and paths handling in mountRepository, and are applicable to the current code and layout.

On the todo list:

• [x] test dir locations repo actions;
• [x] test ssh locations;
• [ ] test remote mounted locations (usb, cifs, s3fs);
  • [x] usb
  • [ ] cifs
  • [ ] s3fs
• [ ] add tasks for an upgrade path from the current to the new layout;
  • if a repo exists at <mount point | location>
    • create a @server directory
    • move borg files to the @server directory
    • clean the borg cache in /root/.cache/borg (to avoid the prompt that the repo was known to be at a different location during backup actions)

[arodier]

I will investigate this one this week. Also, I will set-up something to test backups targets in continuous integration.

[fredericmoulins]

* add tasks for an upgrade path from the current to the new layout;

I have been stuck with this because I couldn't see a way to handle the repository in the ssh:// case.

For the other protocols, it is a question of replicating the logic of mountRepository to get the paths and then check for Borg repositories once the install-protocol-*.yml are done. It is complicated but should be doable.

For the ssh:// case, the access with the backup key is supposed to be restricted to the borg serve RPC. I don't think it is possible to move the repository in a new directory using this RPC.

Maybe the /@server part should be added by the ansible install-protocol-*.yml and not be hardcoded in the Borg wrapper. That way:

• the URL in the backup config file is explicit about where the server backup is;
• the remote ssh:// location (without /@server) can be tested for a repository, and if one exists, the /@server part can be omitted;
• other types of locations can be tested for a repository, and if one exists, moved to a new @server/ directory and the /@server part added to a repository.
• for new installs or new backup locations, the server backup would be located in a @server directory in any case (even ssh://).

I might try to implement this.

[arodier]

Let's talk about this this weekend.

[progmaticltd]

Re-open if needed, please.