xcp icon indicating copy to clipboard operation
xcp copied to clipboard

Add checksum verification feature

Open felix068 opened this issue 2 months ago • 11 comments

Implements feature request #71 for optional checksum verification during file copies to detect hardware errors (memory/storage issues).

Implementation:

  • Uses xxHash64 for fast non-cryptographic checksumming
  • Calculates checksum during copy (zero overhead on source read)
  • Verifies by re-reading destination file only
  • Returns immediate error on mismatch (no retry)
  • Works with both parfile and parblock drivers
  • Thread-safe using Mutex for final checksum storage

Changes:

  • Add --verify-checksum CLI flag
  • Add Config.verify_checksum field
  • Add XcpError::ChecksumMismatch error type
  • Update CopyHandle to compute and verify checksums
  • Add xxhash-rust dependency

Tests:

  • 28 comprehensive test cases covering:
    • Empty, small, and large files
    • Binary patterns (zeros, random, alternating)
    • Recursive directory copies
    • Multiple files
    • Both drivers (parfile/parblock)
    • Various block sizes and worker counts
    • Sparse files
    • File overwriting

Performance:

  • ~2x overhead due to destination re-read (e.g., 34ms → 70ms for 50MB)
  • Acceptable trade-off for critical data integrity

Addresses feedback from @tarka, @Kalinda-Myriad, and @OndrikB in issue #71.

felix068 avatar Oct 06 '25 14:10 felix068