rsync icon indicating copy to clipboard operation
rsync copied to clipboard

--stats flag does not count properly

Open mgutt opened this issue 3 years ago • 3 comments

I mounted an external SMB share as a local path and synced this path through rsync with verbose and stats flags as follows:

rsync -av --stats "${source_path}" "${backup_path}"

Then rsync returned this error:

rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1189) [sender=3.1.3]

So I checked the verbose output and found a permission problem:

DESKTOP-I0HHMD9_Downloads/FileBot_4.9.0_x64.msi
DESKTOP-I0HHMD9_Downloads/FileBot_4.9.1_x64.msi
DESKTOP-I0HHMD9_Downloads/FileZilla_Pro_3.49.2_win64-setup.exe
DESKTOP-I0HHMD9_Downloads/FileZilla_Pro_3.50.0_win64-setup.exe
rsync: send_files failed to open "/mnt/disks/DESKTOP-I0HHMD9_Downloads/FileZilla_Pro_3.51.0_win64-setup.exe": Permission denied (13)
DESKTOP-I0HHMD9_Downloads/FilmeMKVsortbyAudioLastFile.txt
DESKTOP-I0HHMD9_Downloads/Firefox Installer.exe
DESKTOP-I0HHMD9_Downloads/Firefox Setup 82.0.2.exe

I checked my destination path and yes, the file is missing: 2020-11-04 23_53_13

But --stats summary says, that all 24,371 files or 22,745 regular files have been created/transferred:

Number of files: 24,371 (reg: 22,745, dir: 1,626)
Number of created files: 24,371 (reg: 22,745, dir: 1,626)
Number of deleted files: 0
Number of regular files transferred: 22,745

What I expect:

Number of files: 24,371 (reg: 22,745, dir: 1,626)
Number of created files: 24,370 (reg: 22,744, dir: 1,626)
Number of deleted files: 0
Number of regular files transferred: 22,744

mgutt avatar Nov 04 '20 23:11 mgutt

Because the sender couldn't open the file, it didn't make it into the file list, so it was not counted at all. So, all those numbers are correct, there's just no mention of "Unreadable files: 1" in the stats at this time.

WayneD avatar Nov 29 '20 18:11 WayneD

But why are they mentioned under "created" although they aren'T?

As another example. A friend had a defective RAID1 and some files were sucessfully transfered, but many of them not. Example:

Files in source:

find /mnt/nvme0n1p1 -type f | wc -l
9137

Files in destination:

find /mnt/disk2/nvme0n1p1_backup -type f | wc -l
4646

rsync stats:

Number of files: 9,377 (reg: 9,137, dir: 240)
Number of created files: 9,377 (reg: 9,137, dir: 240)
Number of deleted files: 0
Number of regular files transferred: 13,628
Total file size: 315,018,363,492 bytes
Total transferred file size: 602,891,660,116 bytes
Literal data: 602,891,660,116 bytes
Matched data: 0 bytes
File list size: 131,056
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 603,039,596,692
Total bytes received: 272,518

sent 603,039,596,692 bytes  received 272,518 bytes  236,439,862.46 bytes/sec
total size is 315,018,363,492  speedup is 0.52
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender=3.2.3]
Script Finished Jan 23, 2021  00:29.13

1.) rsync did not "create" 9377 files (maybe, if you count the created tmp files which were delete afterwards, but should be in a different line I think) 2.) rsync did not "transfer" 13628 files (maybe, if this contains retries, but then this should be in a different line I think)

mgutt avatar Jan 23 '21 09:01 mgutt