rhash
Problems with verifying multiple hash values generated for the same file | 对同一个文件生成多种散列值进行校验产生的问题
Suppose we generate multiple hash values (CRC32, MD5, ED2K, SHA1) for a file and then verify them, for example:
> rhash -CMEH rhash.exe -o check.sum
The content of check.sum is as follows:
rhash.exe 104eb99c 49c833d816e8ff25ce2b78a728c0a082 c4c024518c156003d2af6dabf215776ccf14e7d1 950a2a255f9907ffb3643cf1be20f02d
Then perform verification:
> rhash -c check.sum
Result:OK
--( Verifying check.sum )---------------------------------------------------------
rhash.exe OK
--------------------------------------------------------------------------------
Everything OK
Verify only CRC32:
> rhash -vCc check.sum
Result: also OK
Verify only MD5:
> rhash -vMc check.sum
Result:ERROR
--( Verifying check.sum )---------------------------------------------------------
rhash.exe ERROR, MD5 is 49C833D816E8FF25CE2B78A728C0A082 should be 950a2a255f9907ffb3643cf1be20f02d
--------------------------------------------------------------------------------
Errors Occurred: Errors:1 Miss:0 Success:0 Total:1
Verify only ED2K:
> rhash -vEc check.sum
Result:ERROR
--( Verifying check.sum )---------------------------------------------------------
rhash.exe ERROR, ED2K is 950A2A255F9907FFB3643CF1BE20F02D should be 49c833d816e8ff25ce2b78a728c0a082
--------------------------------------------------------------------------------
Errors Occurred: Errors:1 Miss:0 Success:0 Total:1
Verify only SHA1:
> rhash -Hc check.sum
Result: OK
--( Verifying check.sum )---------------------------------------------------------
rhash.exe OK
--------------------------------------------------------------------------------
Everything OK
Specify verification for CRC32, MD5, ED2K, and SHA1:
> rhash -vCMEHc check.sum
Result: all OK
--( Verifying check.sum )---------------------------------------------------------
rhash.exe OK
--------------------------------------------------------------------------------
Everything OK
Question:
Why do both not specifying any hash algorithm and specifying all corresponding hash algorithms return OK for verification?
If a specific hash algorithm is specified for verification, but the hash value length is the same as another hash type, Rhash misidentifies it. In this example, MD5 and ED2K hash values have the same length. When verifying MD5, Rhash compares it against the ED2K value; when verifying ED2K, it compares against the MD5 value. Strangely, when both MD5 and ED2K are specified simultaneously, they do not report errors against each other.
Can this issue be avoided by changing the output file format?
Compared to Rhash, the hash file format generated by Fsum is much clearer. No matter how many files or hash algorithms are involved, each file and each hash value occupies only one line. Except for the default MD5 algorithm, every hash value explicitly specifies the algorithm.
> fsum -jnc -crc32 -md5 -edonkey -sha1 fsum.exe
7634c61e ?CRC32*fsum.exe
8e685166c1eba689e35967ee1e430f93 *fsum.exe
182f25b55a3f17c0fe2536dc504cde98 ?EDONKEY*fsum.exe
7c414fdc9f3afd80ed3c56aa250e1758a9142f8b ?SHA1*fsum.exe
During verification, regardless of whether the hash values have the same length, there is no misidentification. Lines that do not conform to the standard format are treated as comments and directly ignored.
对同一个文件生成多种散列值进行校验产生的问题
假如对一个文件生成 CRC32 MD5 ED2K SHA1 多种散列值，然后对其进行校验，比如
> rhash -CMEH rhash.exe -o check.sum
check.sum 文件内容如下
rhash.exe 104eb99c 49c833d816e8ff25ce2b78a728c0a082 c4c024518c156003d2af6dabf215776ccf14e7d1 950a2a255f9907ffb3643cf1be20f02d
然后对其进行校验
> rhash -c check.sum
返回结果
--( Verifying check.sum )--------------------------------------------------------- rhash.exe OK -------------------------------------------------------------------------------- Everything OK
仅校验 CRC32
> rhash -vCc check.sum
返回结果也是 OK
仅校验 MD5
> rhash -vMc check.sum
返回结果
--( Verifying check.sum )--------------------------------------------------------- rhash.exe ERROR, MD5 is 49C833D816E8FF25CE2B78A728C0A082 should be 950a2a255f9907ffb3643cf1be20f02d -------------------------------------------------------------------------------- Errors Occurred: Errors:1 Miss:0 Success:0 Total:1
仅校验 ED2K
> rhash -vEc check.sum
返回结果
--( Verifying check.sum )--------------------------------------------------------- rhash.exe ERROR, ED2K is 950A2A255F9907FFB3643CF1BE20F02D should be 49c833d816e8ff25ce2b78a728c0a082 -------------------------------------------------------------------------------- Errors Occurred: Errors:1 Miss:0 Success:0 Total:1
仅校验 SHA1
> rhash -Hc check.sum
返回结果OK
--( Verifying check.sum )--------------------------------------------------------- rhash.exe OK -------------------------------------------------------------------------------- Everything OK
指定校验 CRC32 MD5 ED2K SHA1
> rhash -vCMEHc check.sum
返回结果都 OK
--( Verifying check.sum )--------------------------------------------------------- rhash.exe OK -------------------------------------------------------------------------------- Everything OK
问题： 为什么不指定散列算法和指定所有对应的散列算法所返回的校验结果都是OK， 如果指定校验某一特定的散列算法，但是该散列值的长度如果与另一种散列值的长度相同，Rhash 就会识别错误。在这个例子中， MD5 的散列值与 ED2K 的散列值长度相同，当要校验 MD5 散列值时，Rhash 就会拿 ED2K 的值来比对；当要校验 ED2K 散列值时，Rhash 就会拿 MD5 的值来比对。但是奇怪的是如果同时指定 MD5 和 ED2K 时却不会相互报错。 不知能否从输出文件的格式上来避免这种情况？
与 Rhash 相比，Fsum 所生成的散列值文件格式就很明确，无论有多少文件多少散列算法，每个文件每个散列值都只占一行，并且除默认的 MD5 散列算法除外，每一种散列值都明确写明算法。
> fsum -jnc -crc32 -md5 -edonkey -sha1 fsum.exe
7634c61e ?CRC32*fsum.exe 8e685166c1eba689e35967ee1e430f93 *fsum.exe 182f25b55a3f17c0fe2536dc504cde98 ?EDONKEY*fsum.exe 7c414fdc9f3afd80ed3c56aa250e1758a9142f8b ?SHA1*fsum.exe
当校验时不管散列值长度是否相同都不会认错，对不符合标准格式的行当做注释直接忽略。
This bug is easily confirmed with any set of hash functions whose digests share the same character length. For example, CRC32 & CRC32C (8 chars), MD4, MD5, ED2K & SNEFRU-128 (32 chars), SHA1, BTIH, RIPEMD-160 & HAS-160 (40 chars), TIGER, TTH & AICH (48 chars), SHA-224 & SHA3-224 (56 chars), GOST94, GOST94-CRYPTOPRO, GOST12-256, SHA-256, EDON-R256, SHA3-256 & SNEFRU-256 (64 chars), SHA-384 & SHA3-384 (96 chars), WHIRLPOOL, GOST12-512, SHA-512, EDON-R512 & SHA3-512 (128 chars) etc.
