RapidCRC-Unicode icon indicating copy to clipboard operation
RapidCRC-Unicode copied to clipboard

slow speed compared with HashCheck fork

Open JaegerFox opened this issue 1 year ago • 3 comments

The creation and comparison of checksums is slow; I'm using a fork that includes Blake3.

https://github.com/user-attachments/assets/bd28ecfe-7968-4c4f-bf4b-dc0cd78056b1

https://github.com/user-attachments/assets/a8670e92-aec5-4170-bb7b-92973655f345

JaegerFox avatar Oct 22 '24 00:10 JaegerFox

That looks like none of the instruction sets accerelation is activate? What CPU are you using? It runs at 4000MB/s on my machine.

Here's a benchmark from ZFS that shows just how dramatically the speed of blake3 is impacted by the instruction set it can use

implementation               1k      4k     16k     64k    256k      1m      4m     16m
edonr-generic              2064    2938    3198    3210    2993    3048    2155    2394
skein-generic               829    1046    1096    1096    1132    1125    1127    1121
sha256-generic              415     463     464     403     438     482     479     458
sha256-x64                  444     495     455     516     507     516     527     529
sha256-ssse3                563     673     702     691     698     704     703     709
sha256-avx                  313     338     328     353     356     352     356     336
sha256-avx2                 597     693     748     743     746     734     749     752
sha256-shani               1603    2221    2449    2510    2502    2533    2530    2540
sha512-generic              605     719     738     731     726     751     757     750
sha512-x64                  650     773     819     840     841     835     838     835
sha512-avx                  426     476     541     534     540     551     538     539
sha512-avx2                 834    1093    1164    1187    1192    1199    1184    1172
blake3-generic              153     152     149     151     152     152     152     150
blake3-sse2                 837    2335    2722    2712    2727    2682    2460    2529
blake3-sse41                831    2455    2771    2849    2501    2094    2565    2683
blake3-avx2                 839    2985    5225    5518    5450    5462    4900    4897
blake3-avx512               850    3044    7116    8198    8202    8138    7672    6572

L4cache avatar Nov 01 '24 03:11 L4cache

Intel Celeron 1007U

JaegerFox avatar Nov 02 '24 00:11 JaegerFox

When calculating multiple small files rapidcrc will be slower since it only calculates one file at a time (#207).

OV2 avatar Nov 05 '24 13:11 OV2