AztecProtocol
feat: Native Merkle Trees
This PR is the first of several where we are moving to use native implementations of Merkle Trees for performance reasons. It includes:
- The core tree implementations for both indexed trees and append only trees.
- The introduction of LMDB as a dependency that we retrieve from GIT and build ourselves.
- The creation of a set of RAII wrapper objects around the LMDB concepts.
- The creation of a committed/uncommitted store on top of LMDB used by the trees for state management
Benchmark results
Metrics with a significant change:
- protocol_circuit_simulation_time_in_ms (private-kernel-tail-to-public): 669 (-55%)
- avm_simulation_time_ms (Token:transfer_public): 26.7 (+39%)
Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Proof generation
Each column represents the number of threads used in proof generation.
| Metric | 1 threads | 4 threads | 16 threads | 32 threads | 64 threads |
|---|---|---|---|---|---|
| proof_construction_time_sha256_ms | 5,808 (+1%) | 1,573 | 708 | 749 (-2%) | 777 (+1%) |
| proof_construction_time_sha256_30_ms | 11,506 (-1%) | 3,092 | 1,377 | 1,437 (+1%) | 1,465 |
| proof_construction_time_sha256_100_ms | 44,236 | 11,831 (-1%) | 5,420 (-1%) | 5,681 (+4%) | 5,710 (-2%) |
| proof_construction_time_poseidon_hash_ms | 79.0 (+1%) | 34.0 | 34.0 | 58.0 (-2%) | 88.0 (+1%) |
| proof_construction_time_poseidon_hash_30_ms | 1,534 | 425 (+1%) | 202 | 226 (-1%) | 278 (+3%) |
| proof_construction_time_poseidon_hash_100_ms | 5,668 | 1,514 | 674 (-1%) | 710 (-5%) | 744 |
L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.
| Metric | 4 txs | 8 txs | 16 txs |
|---|---|---|---|
| l1_rollup_calldata_size_in_bytes | 4,324 | 7,844 | 14,852 |
| l1_rollup_calldata_gas | 49,804 | 92,460 | 177,584 |
| l1_rollup_execution_gas | 1,373,832 | 2,107,384 | 3,892,564 |
| l2_block_processing_time_in_ms | 239 (-4%) | 451 (+2%) | 821 (-1%) |
| l2_block_building_time_in_ms | 8,936 (-1%) | 17,526 (+1%) | 34,850 (-1%) |
| l2_block_rollup_simulation_time_in_ms | 8,936 (-1%) | 17,526 (+1%) | 34,850 (-1%) |
| l2_block_public_tx_process_time_in_ms | 7,555 (-1%) | 15,990 | 33,317 (-1%) |
L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 8 txs.
| Metric | 3 blocks | 5 blocks |
|---|---|---|
| node_history_sync_time_in_ms | 2,953 | 3,871 (+2%) |
| node_database_size_in_bytes | 12,623,952 | 16,765,008 |
| pxe_database_size_in_bytes | 16,254 | 26,813 |
Circuits stats
Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.
| Circuit | simulation_time_in_ms | witness_generation_time_in_ms | input_size_in_bytes | output_size_in_bytes | proving_time_in_ms |
|---|---|---|---|---|---|
| private-kernel-init | 92.2 (-1%) | 422 (+12%) | 21,755 | 44,860 | N/A |
| private-kernel-inner | 169 (-1%) | 731 (+9%) | 72,566 | 45,007 | N/A |
| private-kernel-reset-tiny | 468 (+2%) | 888 (+5%) | 65,675 | 44,846 | N/A |
| private-kernel-tail | 196 (+1%) | 159 (+6%) | 50,686 | 52,257 | N/A |
| base-parity | 5.62 (+1%) | N/A | 160 | 96.0 | N/A |
| root-parity | 35.6 (+1%) | N/A | 73,948 | 96.0 | N/A |
| base-rollup | 2,747 (-1%) | N/A | 189,136 | 664 | N/A |
| root-rollup | 40.4 (+1%) | N/A | 58,173 | 716 | N/A |
| public-kernel-setup | 84.0 (+1%) | N/A | 105,085 | 71,222 | N/A |
| public-kernel-app-logic | 95.4 | N/A | 104,911 | 71,222 | N/A |
| public-kernel-tail | 555 (+1%) | N/A | 410,534 | 16,414 | N/A |
| private-kernel-reset-small | 460 (+2%) | N/A | 66,341 | 45,629 | N/A |
| private-kernel-tail-to-public | :warning: 669 (-55%) | 663 (+7%) | 460,796 | 1,825 | N/A |
| public-kernel-teardown | 82.8 (+1%) | N/A | 105,349 | 71,222 | N/A |
| merge-rollup | 20.5 (+2%) | N/A | 38,174 | 664 | N/A |
| undefined | N/A | N/A | N/A | N/A | 77,804 (-5%) |
Stats on running time collected for app circuits
| Function | input_size_in_bytes | output_size_in_bytes | witness_generation_time_in_ms |
|---|---|---|---|
| ContractClassRegisterer:register | 1,344 | 11,731 | 344 |
| ContractInstanceDeployer:deploy | 1,408 | 11,731 | 18.4 (+1%) |
| MultiCallEntrypoint:entrypoint | 1,920 | 11,731 | 407 |
| FeeJuice:deploy | 1,376 | 11,731 | 390 (-2%) |
| SchnorrAccount:constructor | 1,312 | 11,731 | 74.8 |
| SchnorrAccount:entrypoint | 2,304 | 11,731 | 415 (+1%) |
| Token:privately_mint_private_note | 1,280 | 11,731 | 101 (-6%) |
| FPC:fee_entrypoint_public | 1,344 | 11,731 | 29.2 (+14%) |
| Token:transfer | 1,312 | 11,731 | 242 (+6%) |
| Benchmarking:create_note | 1,344 | 11,731 | 89.5 (+4%) |
| SchnorrAccount:verify_private_authwit | 1,280 | 11,731 | 27.9 (+1%) |
| Token:unshield | 1,376 | 11,731 | 535 |
| FPC:fee_entrypoint_private | 1,376 | 11,731 | 712 (+1%) |
AVM Simulation
Time to simulate various public functions in the AVM.
| Function | time_ms | bytecode_size_in_bytes |
|---|---|---|
| FeeJuice:_increase_public_balance | 56.6 (-9%) | 7,739 |
| FeeJuice:set_portal | 12.0 (-16%) | 2,354 |
| Token:constructor | 81.1 (-4%) | 26,051 |
| FPC:constructor | 55.1 (+1%) | 18,001 |
| FeeJuice:mint_public | 39.4 (+2%) | 5,877 |
| Token:mint_public | 70.8 (+3%) | 10,917 |
| Token:assert_minter_and_mint | 40.7 (+1%) | 7,512 |
| AuthRegistry:set_authorized | 37.5 (-21%) | 4,391 |
| FPC:prepare_fee | 227 (-3%) | 7,043 |
| Token:transfer_public | :warning: 26.7 (+39%) | 39,426 |
| FPC:pay_refund | 61.8 (+4%) | 10,234 |
| Benchmarking:increment_balance | 932 | 6,563 |
| Token:_increase_public_balance | 43.8 (+11%) | 8,433 |
| FPC:pay_refund_with_shielded_rebate | 66.2 (+3%) | 10,783 |
Public DB Access
Time to access various public DBs.
| Function | time_ms |
|---|---|
| get-nullifier-index | 0.160 (-3%) |
Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.
| Metric | 1 leaves | 16 leaves | 64 leaves | 128 leaves | 256 leaves | 512 leaves | 1024 leaves |
|---|---|---|---|---|---|---|---|
| batch_insert_into_append_only_tree_16_depth_ms | 2.20 (+1%) | 3.94 | N/A | N/A | N/A | N/A | N/A |
| batch_insert_into_append_only_tree_16_depth_hash_count | 16.8 | 31.7 | N/A | N/A | N/A | N/A | N/A |
| batch_insert_into_append_only_tree_16_depth_hash_ms | 0.115 (+2%) | 0.111 | N/A | N/A | N/A | N/A | N/A |
| batch_insert_into_append_only_tree_32_depth_ms | N/A | N/A | 11.6 | 17.7 (-2%) | 30.4 (-1%) | 60.3 (+3%) | 113 (-3%) |
| batch_insert_into_append_only_tree_32_depth_hash_count | N/A | N/A | 95.9 | 159 | 287 | 543 | 1,055 |
| batch_insert_into_append_only_tree_32_depth_hash_ms | N/A | N/A | 0.111 | 0.103 (-2%) | 0.0993 (-1%) | 0.104 (+2%) | 0.101 (-3%) |
| batch_insert_into_indexed_tree_20_depth_ms | N/A | N/A | 14.5 | 25.9 (+1%) | 42.7 (-2%) | 86.3 (+5%) | 160 (-3%) |
| batch_insert_into_indexed_tree_20_depth_hash_count | N/A | N/A | 109 | 207 | 355 | 691 | 1,363 |
| batch_insert_into_indexed_tree_20_depth_hash_ms | N/A | N/A | 0.109 | 0.104 | 0.100 (-5%) | 0.107 (+5%) | 0.101 (-3%) |
| batch_insert_into_indexed_tree_40_depth_ms | N/A | N/A | 16.6 (+1%) | N/A | N/A | N/A | N/A |
| batch_insert_into_indexed_tree_40_depth_hash_count | N/A | N/A | 132 | N/A | N/A | N/A | N/A |
| batch_insert_into_indexed_tree_40_depth_hash_ms | N/A | N/A | 0.107 (+1%) | N/A | N/A | N/A | N/A |
Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.
| Metric | 0 registered classes | 1 registered classes |
|---|---|---|
| tx_size_in_bytes | 64,779 | 668,997 |
Transaction size based on fee payment method
| Metric | | | - | |
Wow lots of code :) Looks good mostly although I definitely power-read, just some minor comments
