MiSTer_Utils_Natrox
MiSTer_Utils_Natrox copied to clipboard
Natrox
rsync_migrate.txt contains lots of "[1429] file has vanished"
After running migrate.sh overnight, it completed successfully, but looking at rsync_migrate.txt on the new sd card, there were 412 "[1429] file has vanished". For example: 2025/01/04 01:07:57 [1429] file has vanished: "/media/fat/Filters/Scanlines - Standard/Scanlines_10.txt"
I had to go back and do a second pass with rsync: mount /dev/sda /media/usb0 rsync --log-file=/media/usb0/rsync_migrate-2.txt -avhx --progress /media/fat/. /media/usb0/
Sorry for getting to you late. This is an unusual issue, but I think what is happening is that your ext4 controller is resetting. That could be the case if it detects corruption. Although you've completed the rsync, could you try fsck.fat on your old card? It might throw up corruption errors, and that would explain error 1429.
In accordance with #3, I think I could do something similar and also add a resume file if these errors are detected. That way, you can choose to rsync again after completion. To further look into you issue, if you still have the old SD card, I would very much appreciate if you can turn it into an image and compress it. Then, I could try and reproduce the issue myself.
You could also snail-mail it. :V Regardless, for the next version I'll do some hardening.
Another idea, if you could find me a copy of your rsync_migrate.log I can try and implement checking that for errors, and I can put up a dialog saying to restart and run the utility again. It would automatically continue.
Hey, thanks for the script :) I also have problems with errors popping up saying "file has vanished" during rsync. But the files are there, I can stat them, cat them and everything is okay. ExFAT is not the most stable of all filesystems, it seems.
My source card seems clean:
/media/fat/Scripts# fsck.exfat /dev/mmcblk0p1
exfatprogs version : 1.0.4
/dev/mmcblk0p1: clean. directories 2465, files 39755
So I am doing this after stopping the script (I run this via SSH, not via MISTer GUI):
mount /dev/sda1 /tmp/newfat
rsync -rltD --inplace --ignore-errors --no-whole-file --progress /media/fat/ /tmp/newfat/
If I use these rsync options in the script and write the log file while syncing I still run into errors. So it seems not writing the log file during sync fixes the problem.... I do not know if this just a problem for my setup or a problem in the script, but I wanted to note this, in case you guys run into similar problems.
Hi, I'm very sorry I missed the posts and request for additional information on this issue. Here are the rsync log files requested above.
Since this is months ago, and its been running great after the clone plus follow up clean up rsync, I dont know that an fsck would help. Getting you the original sd card image would be tough as well since its about 600gb. Happy to run any additional tests for you though.
Again, feel bad I never saw your requests back in Feb, and love your work, thanks!!!