Issue with initial backup

[leonard-limon]

Hi, I have a problem with the script : I use the script to backup my external hard drive (formatted as exFat) from my Mac (Mac OS 10.13.6) to another hard disk on my Raspberry Pi server (hard disk formatted as ext4). However, to speedup the initial backup process I copy files via USB 3 on my computer (which is way faster than through the network), I then place the copied files to a folder named 2020-01-01-120000 inside the backup folder. When run for the fist time, I expect this backup script to just create a new folder with today's date and hardlink every file in 2020-01-01-120000 to the newly created folder, however the script is copying again every file thus files in 2020-01-01-120000 and in the newly created folder have different inode id, which is not what I expected. I first though it was related to permissions or file attributes (timestamps) so I copied file permissions and attributes from one hard disk to the other but the problem is still there. What is the thing that makes rsync_tmbackup.sh copy every file again instead of hard-linking?

Hope I have been clear enough, Regards

[dchakro]

I had this same problem with ≈1.5TB of data on work server. And I just can't transfer that much data while working from home due to COVID-19 :) I tried what @leonard-limon mentioned and encountered the same problem.

But I figured out this following workaround which seems to work:

# Get to the backup destination with you pre-existing backup
cd /Volumes/BackupDestination/

# rename your old backup folder
mv Folder Folder.bak

# Initnalize an empty directory with rsync_tmbackup.sh
mkdir Folder
cd Folder
mkdir -p -- "." ; touch "./backup.marker"

# Initiate backup
# I've run: chmod +x rsync_tmbackup.sh; mv rsync_tmbackup.sh /usr/local/bin/rsync_tmbackup

rsync_tmbackup ~/Source/Folder/ /Volumes/BackupDestination/Folder/
#[!] As soon as backup starts hit Ctrl + C to stop the backup
Ctrl + C

# remove marker for the in-progress backup
rm backup.inprogress 

# remove the newly created backup folder where rsync_tmbackup.sh started to copy files
# It will be the current data & time
rm -rf <YYY-MM-DD-HHMMSS> 

# Now get your files from your previous backup to the current working directory.
# [!] Use 'cp' if you want to be really safe with your data and then remove the ../Folder.bak after your backup has worked out
mv ../Folder.bak ./ 

# Rename your folder to the YYY-MM-DD-HHMMSS of last backup
# Put yesterday's date up if you don't know/remember/figure out.
mv Folder.bak "2019-09-11-110134"
ln -s "2019-09-11-110134" latest

# Run rync_tmbackup.sh
rsync_tmbackup ~/Source/Folder/ /.

# rync_tmbackup.sh will go through listing every file, but if you monitor your network/disk usage you'll see that you're not transferring files and in the ouput you'll mostly see something like:
# >f..tp...... /your/files/here.txt

# From next time onwards you just need to run:
rsync_tmbackup ~/Source/Folder/ /Volumes/BackupDestination/Folder/

[leonard-limon]

Thanks for that workaround, I'll try that !