rdfind icon indicating copy to clipboard operation
rdfind copied to clipboard

Faster hashes

Open ido opened this issue 1 year ago • 0 comments

If you check the entire file's contents after comparing hashes, then it is probably worthwhile to explore faster hashes such as xxhash or to use hardware-accelerated checksums such as CRC32 on x86_64 if it's available, as a first pass before more expensive hashes or block by block file comparisons.

Here's a benchmark of fast hashes: https://github.com/rurban/smhasher#summary

Here's the crc32c Intel intrinsics: https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#ig_expand=1494,1494&text=crc32

ido avatar Oct 07 '23 03:10 ido