updating an existing extraction to match a more recent archive's content

[Benouare]

Hello,

First of all, thx for this project. That is really usefull and secure!

My use case. I want to migrate my client from an hardware to a other one. And dont want to stop for a long time my service. So i would like to :

• 3 days before the migration : make a borg extract from the last archive ( Let's say "archive_A", for example 800 Go ) on the new hardware , to get 90% of the data,
• 1 day before the migration : cut/stop the main service,
• DDay +1h : let the last borg create going on. Let's say it will make "archive_E" with 10 Go more.
• DDay +10h : apply the "diff" between Archive A and E, so apply one by one, or all in once, archive B, archive C, D, E on the current borg extract target path. => so now i will have 100% of the data.

What i did : For the moment, i think i cant do what i want. If i make borg extract archive_A and after the end, I redo/remake the command, it' goes/restart at the beggining (i would like that borg says "Hey dude, there is nothing to update, your target is already at the last version, so i will do nothing" )

What i want : I want more or less to pull only the files that i dont currently have in my target borg extract dir from an archive.

superman@kwqyzfkp:~$ borg extract --progress  ssh://[email protected]/./repo::AS6102T-5A17-2022-12-31T23:02:26
# Here the extract ends succesfully, i "restart" it, and would like to get a message  as "Nothing to do", because no diff.  
superman@kwqyzfkp:~$ borg extract --progress  ssh://[email protected]/./repo::AS6102T-5A17-2022-12-31T23:02:26
  3.3% Extracting: var/www/drive/data/ouichelorraine/files/Photos/2022/06/22-06-24 12-07-25 2EF2.jpg
# Borg is not checking / comparing the extract dir with the current/last archive.  So it's starting a full dl/extract. It's not what i want.... 

Cheers (& merry Christmas)

[ThomasWaldmann]

borg extract for a long time did not support "incremental extraction" at all: it assumed that you are extracting into an empty directory (and if it was not empty, it usually removed existing files of same name before extracting them, to make sure the file metadata and data is exactly as in the archive).

I recently changed this a bit so it does not kill the existing file if the fs files' path, size and timestamp is exactly as the one in the archive. IIRC I only changed this in master branch, which will become borg2 at some time (not released yet).

So, guess for your use case it is better if you use rsync for the "migration syncing" and borg for backups.

[ThomasWaldmann]

BTW, this is the wrong issue tracker, I will transfer it to borgbackup/borg ...

[ThomasWaldmann]

@Benouare let's continue here.

[ThomasWaldmann]

BTW, theoretically it would be possible to use rsync from the borgfs created by borg mount repo::last_archive to your destination directory, but that has some issues:

• might need lots of memory, depending on your archive size
• might be slower than borg extract
• borgfs does not support all metadata (e.g. no ACLs)

[Benouare]

BTW, theoretically it would be possible to use rsync from the borgfs created by borg mount repo::last_archive to your destination directory, but that has some issues:

• might need lots of memory, depending on your archive size
• might be slower than borg extract
• borgfs does not support all metadata (e.g. no ACLs)

Hi, will try this asap and give you a feedback. It's seems to be good enought in my case. Thx for the tips.

[ThomasWaldmann]

IIRC we already have at least one other ticket for an "updating extraction".