SHH rsync will only copy to one drive then fail when the drive is full, even if other drives have space

[Betonhaus]

Describe the bug

If you SSH rsync transfer from an old server to a mergerfs pool on a new server, it will only copy the files to one drive, and when that drive is full the file downloads will fail with an "insufficient space" error messages - even if the mergerfs pool has more drives lit lots of space

To Reproduce

from old server, transfer by rsync to new server a volume of data that is more then any single drive in the mergerfs pool rsync -avz /local/directory user@remote_server:/remote/mergerfs/pool

Expected behavior

option 1: mergerfs distributes the files between drives somewhat evenly option 2: when the first drive reaches it's minimum free space limit, switch to the next drive to continue the transfer

System information:

• OS, kernel version: uname -a Linux archie 6.8.0-47-generic #47-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 27 21:40:26 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

• mergerfs version: mergerfs -V mergerfs version: 2.33.5

• mergerfs settings /mnt/dsk/*/ /mnt/pool fuse.mergerfs defaults,nonempty,allow_other,use_ino,cache.files=partial,moveonenospc=true,dropcacheonclose=true,minfreespace=10G,fsname=mergerfs 0 0

• List of drives, filesystems, & sizes:

  • df -h Filesystem Size Used Avail Use% Mounted on tmpfs 6.3G 4.1M 6.3G 1% /run efivarfs 256K 153K 99K 61% /sys/firmware/efi/efivars /dev/nvme1n1p2 442G 44G 376G 11% / tmpfs 32G 0 32G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock mergerfs 51T 25T 24T 51% /mnt/pool /dev/nvme0n1 938G 277G 614G 32% /mnt/wrk /dev/sde1 220G 57G 152G 28% /mnt/win /dev/sdb 13T 5.9T 6.2T 49% /mnt/dsk/2 /dev/sda 13T 5.9T 6.2T 49% /mnt/dsk/1 /dev/sdc 13T 5.8T 6.3T 49% /mnt/dsk/3 /dev/nvme1n1p1 1.1G 4.4M 1.1G 1% /boot/efi /dev/sdd 13T 6.9T 5.2T 58% /mnt/dsk/4 tmpfs 6.3G 8.0K 6.3G 1% /run/user/1000

  • lsblk -f NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS sda ext4 1.0 e5db7a4f-68b7-4b39-8830-75f40d2f4798 6.1T 46% /mnt/dsk/1 sdb ext4 1.0 f28f400e-b27c-46f9-b507-3bd93c310549 6.2T 46% /mnt/dsk/2 sdc ext4 1.0 22b64dd2-4031-4f4a-9b04-15c4b297ba78 6.2T 46% /mnt/dsk/3 sdd ext4 1.0 fbf46f39-443d-4655-b2ad-1c1335b084e4 5.1T 54% /mnt/dsk/4 sde
    └─sde1 ext4 1.0 0a407b18-8017-4225-aa02-5f34f23cc26a 151.5G 26% /mnt/win nvme0n1 ext4 1.0 c59f723c-0579-45ec-a4ff-5d914c744c44 609.2G 30% /mnt/wrk nvme1n1
    ├─nvme1n1p1 vfat FAT32 9597-8CEA 1G 0% /boot/efi ├─nvme1n1p2 ext4 1.0 75f48842-bf3a-482d-9cf3-4f213a1b038c 375.8G 10% / └─nvme1n1p3 swap 1 3400038d-8866-4d72-ae6b-8f40746d1acb [SWAP]