btcd
btcd copied to clipboard
blockchain: rolling merkle root
This PR adds a new RollingMerkleTree type for efficiently computing merkle roots. The leaves
of the tree are iteratively pushed onto the tree, and any complete substrees are evaluated and
stored so that subsequent pushes may consume them when their sibling tree is complete. As a
result, this requires O(log n) additional memory be allocated. The current BuildMerkleTreeStore
allocates an array of up to 4 times the number of leaves, which is used to store internal nodes and the root.
For 2000 leaves, the benchmarks show that this method is about 7% slower (~90 microseconds), but uses 99.04% less memory and 99.85% fewer allocations. These allocations reduce gc pressure, especially during IBD when the large slices of hashes currently used are almost immediately discarded.
BenchmarkMerkle/rolling-1000-8 2000 694382 ns/op 858 B/op 3 allocs/op
BenchmarkMerkle/rolling-2000-8 1000 1367763 ns/op 923 B/op 3 allocs/op
BenchmarkMerkle/rolling-4000-8 500 2900103 ns/op 1210 B/op 6 allocs/op
BenchmarkMerkle/rolling-8000-8 300 5799806 ns/op 2034 B/op 10 allocs/op
BenchmarkMerkle/rolling-16000-8 100 11572403 ns/op 1578 B/op 3 allocs/op
BenchmarkMerkle/rolling-32000-8 50 22949069 ns/op 1720 B/op 5 allocs/op
BenchmarkMerkle/old-1000-8 2000 655435 ns/op 48416 B/op 1002 allocs/op
BenchmarkMerkle/old-2000-8 1000 1277192 ns/op 96800 B/op 2002 allocs/op
BenchmarkMerkle/old-4000-8 500 2533821 ns/op 193568 B/op 4002 allocs/op
BenchmarkMerkle/old-8000-8 300 5029310 ns/op 387104 B/op 8002 allocs/op
BenchmarkMerkle/old-16000-8 200 10135631 ns/op 774176 B/op 16002 allocs/op
BenchmarkMerkle/old-32000-8 100 20442194 ns/op 1548340 B/op 32002 allocs/op
alternative to #1377
@jcvernaleo (as per #1530)
- Low priority
- Enhancement
- Duplicate (of #1377, not sure if we have a tag for this)
@cfromknecht mind rebasing so the actions can run?