tarka
[Feature Request] checksum verification
Wondering if it's possible to add checksum verification or if xcp's parallelization prevents this?
By my testing xcp is by far the most performant option of the CP replacements.
However, https://github.com/thewh1teagle/mc (which is overall pretty incomplete) allows for checksum verification.
Crucially, the implementation doesn't require the file to be re-read from the sources, as the checksum is calculated during the initial transfer, and then compared when the destination file is read back.
Would absolutely love if this were to happen.
I've considered this, but I'm not sure what the failure response should be. mc seems to just throw an error with no attempt at correction. This is probably correct, as with machine-local copy mismatches the only culprits I can think of would be storage or memory errors, in which case the whole machine is suspect and any attempt at correction would be pointless. I'd be interested to hear what people's use-cases are for this though.
(Passing thought: f you need performance but also belt and braces you could use xcp to perform the copy with the parblock driver and then run rsync afterwards to detect/correct any errors.)
As it was requested to give my inputs / usecases: I would be interested in such a feature as checksumming. I agree that by default, the behaviour should be throwing an error with no attempt at correction for the reasons stated.
In general, my use case is that I frequently use terminal commands to perform routine backups of important files, and checksumming being built into some tools like xcp would be really nice for knowing if there were any checksum errors at all, and alerts me as the user to needing to investigate further.
I was looking for an alternative to the GNU copy command, and this feature being missing is pretty much what's stopping me from using this tool.
In my opinion, checksum verification would be useful for copying between drives, or just for peace of mind. Personally, I'm a bit paranoid even copying or moving files from one place on my drive to another, and knowing that my files are still the same (barring extreme force majeure) is definitely very nice.
As for the failure response, I considered for a bit to suggest having it be configurable (maybe just throw an error on default, with a configurable --retries argument?), but if that was implemented, it should be accompanied with appropriate warnings that checksum failures may be the sign of a failing drive and that retrying the write might only cause more damage (maybe even force an interaction confirming that the user wants to retry and is aware of the potential consequences.)
I've created a PR for this feature in case you're interested. Can be modified and debated of course...
