Add Blake3 as new hash algorithm
Blake3 is a modern cryptographic hash function designed for exceptional speed, parallelism and security. Compared to traditional algorithms like SHA-256 and SHA-512, Blake3 offers significantly faster performance as we can see in the following image shared in the git repository (10x faster than SHA). At the same time Blake3 maintain cryptographic strength suitable for non collision critical scenarios such as Merkle Tree construction, intermediate state hashing or chaching.
Given that Neo targets both .Net standard 2.1 and .NET 9, I propose introducing Blake3 support conditionally, limited to .NET 9.
Meanwhile NEO Core libraries targeting .NET Standard 2.1 will maintain compatibility by falling back to SHA-256 or SHA-512 which remain secure and widely supported.
This hybrid approach ensures:
- Performance gains
- Maximum compatibility
- Future scalability
Meanwhile NEO Core libraries targeting .NET Standard 2.1 will maintain compatibility by falling back to SHA-256 or SHA-512 which remain secure and widely supported.
Whole the project must use the same algorithm, there is any Net Standard 2.1 library available?
For now, there is no version compatible with .NET Standard 2.1, but it could be used for non-sensitive functionalities such as caching information, etc.
Also, I will open a new issue to discuss the use of .NET standard 2.1 since no new versions of .net stander will be released
- It's very hard to change things used for merkle, tx/account/network payload hashes. We can't even solve https://github.com/neo-project/neo/issues/938.
- Hashes are not the main performance problem.
- SHA256 is pretty fast on its own and we even use double SHA256 in some cases.
- Blake3 is not widely used/available (at least compared to ubiquitous SHA256).
I've quickly tried the most widely used Go implementation of it (https://pkg.go.dev/lukechampine.com/blake) with Merkle tree benchmark we already have. It uses double SHA256 originally, changing to a single Blake3 results in this:
goos: linux
goarch: amd64
pkg: github.com/nspcc-dev/neo-go/pkg/crypto/hash
cpu: AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics
│ merkle.doublesha256 │ merkle.blake3 │
│ sec/op │ sec/op vs base │
Merkle/NewMerkleTree-16 33.92m ± 3% 29.25m ± 1% -13.75% (p=0.000 n=10)
Merkle/CalcMerkleRoot-16 15.813m ± 1% 9.618m ± 2% -39.18% (p=0.000 n=10)
geomean 23.16m 16.77m -27.57%
│ merkle.doublesha256 │ merkle.blake3 │
│ B/op │ B/op vs base │
Merkle/NewMerkleTree-16 19.86Mi ± 0% 22.91Mi ± 0% +15.37% (p=0.000 n=10)
Merkle/CalcMerkleRoot-16 0.000Ki ± 0% 1.062Ki ± 0% ? (p=0.000 n=10)
geomean ¹ 157.9Ki ?
¹ summaries must be >0 to compute geomean
│ merkle.doublesha256 │ merkle.blake3 │
│ allocs/op │ allocs/op vs base │
Merkle/NewMerkleTree-16 300.0k ± 0% 400.0k ± 0% +33.33% (p=0.000 n=10)
Merkle/CalcMerkleRoot-16 0.00 ± 0% 17.00 ± 0% ? (p=0.000 n=10)
geomean ¹ 2.608k ?
¹ summaries must be >0 to compute geomean
while changing to a single SHA256 yields this:
goos: linux
goarch: amd64
pkg: github.com/nspcc-dev/neo-go/pkg/crypto/hash
cpu: AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics
│ merkle.doublesha256 │ merkle.sha256 │
│ sec/op │ sec/op vs base │
Merkle/NewMerkleTree-16 33.92m ± 3% 25.75m ± 4% -24.07% (p=0.000 n=10)
Merkle/CalcMerkleRoot-16 15.813m ± 1% 9.632m ± 1% -39.09% (p=0.000 n=10)
geomean 23.16m 15.75m -31.99%
│ merkle.doublesha256 │ merkle.sha256 │
│ B/op │ B/op vs base │
Merkle/NewMerkleTree-16 19.86Mi ± 0% 19.86Mi ± 0% ~ (p=0.897 n=10)
Merkle/CalcMerkleRoot-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
geomean ² +0.00% ²
¹ all samples are equal
² summaries must be >0 to compute geomean
│ merkle.doublesha256 │ merkle.sha256 │
│ allocs/op │ allocs/op vs base │
Merkle/NewMerkleTree-16 300.0k ± 0% 300.0k ± 0% ~ (p=1.000 n=10)
Merkle/CalcMerkleRoot-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
geomean ² +0.00% ²
¹ all samples are equal
² summaries must be >0 to compute geomean
Some test data in the above image are not reliable. For example, If acceleration instructions are used, SHA will be significantly faster than plain implementation.