lodestar
lodestar copied to clipboard
Remove testcontainers dependency for web3signer test
Motivation
- PR https://github.com/ChainSafe/lodestar/pull/4502 introduced a dev testcontainers since @dadepo was having issues with docker
I really want to start decreasing our dependencies exposure, not increasing it unless required. We have rolled our docker runners in the past and should continue to do so.
Description
- Use adhoc docker
@dadepo I've removed the use of --network=host. If you have to run this tests locally try to understand the incompatibilities and fix them together without 3rd party libraries
Performance Report
✔️ no performance regression detected
Full benchmark results
| Benchmark suite | Current: 9ef50342c0d421e3d9f846f4017331970b203bae | Previous: 8709e3a7a718e634dcc897414f7595bc97b6cd9e | Ratio |
|---|---|---|---|
| getPubkeys - index2pubkey - req 1000 vs - 250000 vc | 2.0222 ms/op | 2.2726 ms/op | 0.89 |
| getPubkeys - validatorsArr - req 1000 vs - 250000 vc | 64.627 us/op | 85.034 us/op | 0.76 |
| BLS verify - blst-native | 1.6398 ms/op | 1.8825 ms/op | 0.87 |
| BLS verifyMultipleSignatures 3 - blst-native | 3.3554 ms/op | 3.8568 ms/op | 0.87 |
| BLS verifyMultipleSignatures 8 - blst-native | 7.2232 ms/op | 8.3030 ms/op | 0.87 |
| BLS verifyMultipleSignatures 32 - blst-native | 26.180 ms/op | 30.109 ms/op | 0.87 |
| BLS aggregatePubkeys 32 - blst-native | 34.879 us/op | 39.878 us/op | 0.87 |
| BLS aggregatePubkeys 128 - blst-native | 135.23 us/op | 154.58 us/op | 0.87 |
| getAttestationsForBlock | 82.174 ms/op | 98.900 ms/op | 0.83 |
| isKnown best case - 1 super set check | 435.00 ns/op | 440.00 ns/op | 0.99 |
| isKnown normal case - 2 super set checks | 425.00 ns/op | 419.00 ns/op | 1.01 |
| isKnown worse case - 16 super set checks | 424.00 ns/op | 420.00 ns/op | 1.01 |
| CheckpointStateCache - add get delete | 8.9270 us/op | 9.4480 us/op | 0.94 |
| validate gossip signedAggregateAndProof - struct | 4.2520 ms/op | 4.3875 ms/op | 0.97 |
| validate gossip attestation - struct | 2.0347 ms/op | 2.0799 ms/op | 0.98 |
| pickEth1Vote - no votes | 2.1913 ms/op | 2.9606 ms/op | 0.74 |
| pickEth1Vote - max votes | 19.643 ms/op | 25.381 ms/op | 0.77 |
| pickEth1Vote - Eth1Data hashTreeRoot value x2048 | 10.082 ms/op | 11.991 ms/op | 0.84 |
| pickEth1Vote - Eth1Data hashTreeRoot tree x2048 | 18.514 ms/op | 23.462 ms/op | 0.79 |
| pickEth1Vote - Eth1Data fastSerialize value x2048 | 1.3556 ms/op | 1.6052 ms/op | 0.84 |
| pickEth1Vote - Eth1Data fastSerialize tree x2048 | 13.362 ms/op | 16.664 ms/op | 0.80 |
| bytes32 toHexString | 965.00 ns/op | 1.2770 us/op | 0.76 |
| bytes32 Buffer.toString(hex) | 634.00 ns/op | 767.00 ns/op | 0.83 |
| bytes32 Buffer.toString(hex) from Uint8Array | 852.00 ns/op | 1.0340 us/op | 0.82 |
| bytes32 Buffer.toString(hex) + 0x | 631.00 ns/op | 789.00 ns/op | 0.80 |
| Object access 1 prop | 0.30800 ns/op | 0.42600 ns/op | 0.72 |
| Map access 1 prop | 0.25700 ns/op | 0.28900 ns/op | 0.89 |
| Object get x1000 | 15.999 ns/op | 17.974 ns/op | 0.89 |
| Map get x1000 | 0.89200 ns/op | 0.99200 ns/op | 0.90 |
| Object set x1000 | 105.45 ns/op | 134.97 ns/op | 0.78 |
| Map set x1000 | 63.959 ns/op | 84.185 ns/op | 0.76 |
| Return object 10000 times | 0.33380 ns/op | 0.37530 ns/op | 0.89 |
| Throw Error 10000 times | 5.1964 us/op | 5.9733 us/op | 0.87 |
| enrSubnets - fastDeserialize 64 bits | 2.3550 us/op | 3.2570 us/op | 0.72 |
| enrSubnets - ssz BitVector 64 bits | 661.00 ns/op | 834.00 ns/op | 0.79 |
| enrSubnets - fastDeserialize 4 bits | 352.00 ns/op | 431.00 ns/op | 0.82 |
| enrSubnets - ssz BitVector 4 bits | 669.00 ns/op | 858.00 ns/op | 0.78 |
| prioritizePeers score -10:0 att 32-0.1 sync 2-0 | 81.743 us/op | 106.93 us/op | 0.76 |
| prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 | 117.89 us/op | 142.90 us/op | 0.82 |
| prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 | 195.46 us/op | 253.84 us/op | 0.77 |
| prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 | 341.72 us/op | 392.85 us/op | 0.87 |
| prioritizePeers score 0:0 att 64-1 sync 4-1 | 407.04 us/op | 464.72 us/op | 0.88 |
| RateTracker 1000000 limit, 1 obj count per request | 180.84 ns/op | 214.54 ns/op | 0.84 |
| RateTracker 1000000 limit, 2 obj count per request | 141.45 ns/op | 167.47 ns/op | 0.84 |
| RateTracker 1000000 limit, 4 obj count per request | 122.22 ns/op | 146.13 ns/op | 0.84 |
| RateTracker 1000000 limit, 8 obj count per request | 112.73 ns/op | 132.82 ns/op | 0.85 |
| RateTracker with prune | 4.2450 us/op | 5.7180 us/op | 0.74 |
| array of 16000 items push then shift | 2.7287 us/op | 3.2262 us/op | 0.85 |
| LinkedList of 16000 items push then shift | 15.950 ns/op | 18.030 ns/op | 0.88 |
| array of 16000 items push then pop | 226.01 ns/op | 266.67 ns/op | 0.85 |
| LinkedList of 16000 items push then pop | 14.858 ns/op | 17.137 ns/op | 0.87 |
| array of 24000 items push then shift | 4.0358 us/op | 4.6542 us/op | 0.87 |
| LinkedList of 24000 items push then shift | 18.618 ns/op | 21.458 ns/op | 0.87 |
| array of 24000 items push then pop | 203.86 ns/op | 236.47 ns/op | 0.86 |
| LinkedList of 24000 items push then pop | 18.274 ns/op | 18.616 ns/op | 0.98 |
| intersect bitArray bitLen 8 | 11.770 ns/op | 11.679 ns/op | 1.01 |
| intersect array and set length 8 | 174.24 ns/op | 199.97 ns/op | 0.87 |
| intersect bitArray bitLen 128 | 61.923 ns/op | 72.499 ns/op | 0.85 |
| intersect array and set length 128 | 2.3202 us/op | 2.4437 us/op | 0.95 |
| Buffer.concat 32 items | 2.2380 ns/op | 1.9930 ns/op | 1.12 |
| pass gossip attestations to forkchoice per slot | 3.1519 ms/op | 3.6377 ms/op | 0.87 |
| computeDeltas | 3.4072 ms/op | 3.1617 ms/op | 1.08 |
| computeProposerBoostScoreFromBalances | 812.76 us/op | 914.51 us/op | 0.89 |
| altair processAttestation - 250000 vs - 7PWei normalcase | 3.2025 ms/op | 4.4580 ms/op | 0.72 |
| altair processAttestation - 250000 vs - 7PWei worstcase | 4.9510 ms/op | 6.7126 ms/op | 0.74 |
| altair processAttestation - setStatus - 1/6 committees join | 185.69 us/op | 228.91 us/op | 0.81 |
| altair processAttestation - setStatus - 1/3 committees join | 355.94 us/op | 425.20 us/op | 0.84 |
| altair processAttestation - setStatus - 1/2 committees join | 556.44 us/op | 594.43 us/op | 0.94 |
| altair processAttestation - setStatus - 2/3 committees join | 727.53 us/op | 801.28 us/op | 0.91 |
| altair processAttestation - setStatus - 4/5 committees join | 991.44 us/op | 1.0664 ms/op | 0.93 |
| altair processAttestation - setStatus - 100% committees join | 1.1714 ms/op | 1.2616 ms/op | 0.93 |
| altair processBlock - 250000 vs - 7PWei normalcase | 25.769 ms/op | 30.163 ms/op | 0.85 |
| altair processBlock - 250000 vs - 7PWei normalcase hashState | 39.630 ms/op | 37.941 ms/op | 1.04 |
| altair processBlock - 250000 vs - 7PWei worstcase | 75.837 ms/op | 93.205 ms/op | 0.81 |
| altair processBlock - 250000 vs - 7PWei worstcase hashState | 106.22 ms/op | 99.096 ms/op | 1.07 |
| phase0 processBlock - 250000 vs - 7PWei normalcase | 3.4381 ms/op | 4.2678 ms/op | 0.81 |
| phase0 processBlock - 250000 vs - 7PWei worstcase | 45.243 ms/op | 48.020 ms/op | 0.94 |
| altair processEth1Data - 250000 vs - 7PWei normalcase | 770.12 us/op | 1.0748 ms/op | 0.72 |
| Tree 40 250000 create | 754.12 ms/op | 1.1181 s/op | 0.67 |
| Tree 40 250000 get(125000) | 253.01 ns/op | 294.40 ns/op | 0.86 |
| Tree 40 250000 set(125000) | 2.4956 us/op | 3.5923 us/op | 0.69 |
| Tree 40 250000 toArray() | 30.045 ms/op | 33.267 ms/op | 0.90 |
| Tree 40 250000 iterate all - toArray() + loop | 30.566 ms/op | 32.901 ms/op | 0.93 |
| Tree 40 250000 iterate all - get(i) | 100.42 ms/op | 112.08 ms/op | 0.90 |
| MutableVector 250000 create | 16.145 ms/op | 16.934 ms/op | 0.95 |
| MutableVector 250000 get(125000) | 13.110 ns/op | 14.920 ns/op | 0.88 |
| MutableVector 250000 set(125000) | 642.38 ns/op | 907.45 ns/op | 0.71 |
| MutableVector 250000 toArray() | 7.2206 ms/op | 7.9888 ms/op | 0.90 |
| MutableVector 250000 iterate all - toArray() + loop | 7.3013 ms/op | 8.1623 ms/op | 0.89 |
| MutableVector 250000 iterate all - get(i) | 3.4414 ms/op | 3.4788 ms/op | 0.99 |
| Array 250000 create | 6.8358 ms/op | 7.3719 ms/op | 0.93 |
| Array 250000 clone - spread | 4.0637 ms/op | 5.7128 ms/op | 0.71 |
| Array 250000 get(125000) | 1.7400 ns/op | 1.7920 ns/op | 0.97 |
| Array 250000 set(125000) | 1.7370 ns/op | 1.7590 ns/op | 0.99 |
| Array 250000 iterate all - loop | 170.39 us/op | 170.56 us/op | 1.00 |
| effectiveBalanceIncrements clone Uint8Array 300000 | 84.277 us/op | 93.430 us/op | 0.90 |
| effectiveBalanceIncrements clone MutableVector 300000 | 706.00 ns/op | 1.3360 us/op | 0.53 |
| effectiveBalanceIncrements rw all Uint8Array 300000 | 252.38 us/op | 254.72 us/op | 0.99 |
| effectiveBalanceIncrements rw all MutableVector 300000 | 156.87 ms/op | 240.64 ms/op | 0.65 |
| phase0 afterProcessEpoch - 250000 vs - 7PWei | 197.08 ms/op | 202.99 ms/op | 0.97 |
| phase0 beforeProcessEpoch - 250000 vs - 7PWei | 71.213 ms/op | 77.585 ms/op | 0.92 |
| altair processEpoch - mainnet_e81889 | 554.07 ms/op | 608.57 ms/op | 0.91 |
| mainnet_e81889 - altair beforeProcessEpoch | 140.93 ms/op | 150.38 ms/op | 0.94 |
| mainnet_e81889 - altair processJustificationAndFinalization | 23.247 us/op | 25.842 us/op | 0.90 |
| mainnet_e81889 - altair processInactivityUpdates | 10.422 ms/op | 10.876 ms/op | 0.96 |
| mainnet_e81889 - altair processRewardsAndPenalties | 92.069 ms/op | 97.267 ms/op | 0.95 |
| mainnet_e81889 - altair processRegistryUpdates | 4.8210 us/op | 4.5440 us/op | 1.06 |
| mainnet_e81889 - altair processSlashings | 913.00 ns/op | 1.0210 us/op | 0.89 |
| mainnet_e81889 - altair processEth1DataReset | 984.00 ns/op | 1.0870 us/op | 0.91 |
| mainnet_e81889 - altair processEffectiveBalanceUpdates | 2.4372 ms/op | 2.3044 ms/op | 1.06 |
| mainnet_e81889 - altair processSlashingsReset | 6.7470 us/op | 7.0060 us/op | 0.96 |
| mainnet_e81889 - altair processRandaoMixesReset | 7.3830 us/op | 7.8360 us/op | 0.94 |
| mainnet_e81889 - altair processHistoricalRootsUpdate | 830.00 ns/op | 993.00 ns/op | 0.84 |
| mainnet_e81889 - altair processParticipationFlagUpdates | 3.5930 us/op | 4.2250 us/op | 0.85 |
| mainnet_e81889 - altair processSyncCommitteeUpdates | 860.00 ns/op | 861.00 ns/op | 1.00 |
| mainnet_e81889 - altair afterProcessEpoch | 174.50 ms/op | 200.51 ms/op | 0.87 |
| phase0 processEpoch - mainnet_e58758 | 501.60 ms/op | 549.18 ms/op | 0.91 |
| mainnet_e58758 - phase0 beforeProcessEpoch | 222.04 ms/op | 247.21 ms/op | 0.90 |
| mainnet_e58758 - phase0 processJustificationAndFinalization | 26.124 us/op | 26.690 us/op | 0.98 |
| mainnet_e58758 - phase0 processRewardsAndPenalties | 134.02 ms/op | 146.34 ms/op | 0.92 |
| mainnet_e58758 - phase0 processRegistryUpdates | 12.292 us/op | 12.111 us/op | 1.01 |
| mainnet_e58758 - phase0 processSlashings | 1.0840 us/op | 903.00 ns/op | 1.20 |
| mainnet_e58758 - phase0 processEth1DataReset | 1.0710 us/op | 952.00 ns/op | 1.13 |
| mainnet_e58758 - phase0 processEffectiveBalanceUpdates | 2.1569 ms/op | 2.1789 ms/op | 0.99 |
| mainnet_e58758 - phase0 processSlashingsReset | 7.1740 us/op | 6.5810 us/op | 1.09 |
| mainnet_e58758 - phase0 processRandaoMixesReset | 8.5640 us/op | 7.6350 us/op | 1.12 |
| mainnet_e58758 - phase0 processHistoricalRootsUpdate | 1.4840 us/op | 1.0740 us/op | 1.38 |
| mainnet_e58758 - phase0 processParticipationRecordUpdates | 5.6790 us/op | 6.5570 us/op | 0.87 |
| mainnet_e58758 - phase0 afterProcessEpoch | 142.14 ms/op | 162.71 ms/op | 0.87 |
| phase0 processEffectiveBalanceUpdates - 250000 normalcase | 2.7685 ms/op | 2.6840 ms/op | 1.03 |
| phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 | 3.4136 ms/op | 3.5223 ms/op | 0.97 |
| altair processInactivityUpdates - 250000 normalcase | 47.456 ms/op | 44.367 ms/op | 1.07 |
| altair processInactivityUpdates - 250000 worstcase | 38.274 ms/op | 54.059 ms/op | 0.71 |
| phase0 processRegistryUpdates - 250000 normalcase | 10.520 us/op | 10.129 us/op | 1.04 |
| phase0 processRegistryUpdates - 250000 badcase_full_deposits | 480.08 us/op | 440.28 us/op | 1.09 |
| phase0 processRegistryUpdates - 250000 worstcase 0.5 | 207.71 ms/op | 227.31 ms/op | 0.91 |
| altair processRewardsAndPenalties - 250000 normalcase | 78.602 ms/op | 136.34 ms/op | 0.58 |
| altair processRewardsAndPenalties - 250000 worstcase | 80.028 ms/op | 90.286 ms/op | 0.89 |
| phase0 getAttestationDeltas - 250000 normalcase | 11.628 ms/op | 16.316 ms/op | 0.71 |
| phase0 getAttestationDeltas - 250000 worstcase | 12.177 ms/op | 16.513 ms/op | 0.74 |
| phase0 processSlashings - 250000 worstcase | 5.3292 ms/op | 5.6272 ms/op | 0.95 |
| altair processSyncCommitteeUpdates - 250000 | 269.76 ms/op | 300.73 ms/op | 0.90 |
| BeaconState.hashTreeRoot - No change | 499.00 ns/op | 480.00 ns/op | 1.04 |
| BeaconState.hashTreeRoot - 1 full validator | 57.373 us/op | 63.186 us/op | 0.91 |
| BeaconState.hashTreeRoot - 32 full validator | 572.06 us/op | 784.36 us/op | 0.73 |
| BeaconState.hashTreeRoot - 512 full validator | 7.2641 ms/op | 6.6026 ms/op | 1.10 |
| BeaconState.hashTreeRoot - 1 validator.effectiveBalance | 69.855 us/op | 80.821 us/op | 0.86 |
| BeaconState.hashTreeRoot - 32 validator.effectiveBalance | 1.1557 ms/op | 1.2353 ms/op | 0.94 |
| BeaconState.hashTreeRoot - 512 validator.effectiveBalance | 14.635 ms/op | 17.137 ms/op | 0.85 |
| BeaconState.hashTreeRoot - 1 balances | 57.768 us/op | 53.144 us/op | 1.09 |
| BeaconState.hashTreeRoot - 32 balances | 570.57 us/op | 580.55 us/op | 0.98 |
| BeaconState.hashTreeRoot - 512 balances | 5.4472 ms/op | 6.3355 ms/op | 0.86 |
| BeaconState.hashTreeRoot - 250000 balances | 78.536 ms/op | 96.927 ms/op | 0.81 |
| aggregationBits - 2048 els - zipIndexesInBitList | 34.459 us/op | 43.217 us/op | 0.80 |
| regular array get 100000 times | 67.419 us/op | 68.193 us/op | 0.99 |
| wrappedArray get 100000 times | 67.401 us/op | 68.112 us/op | 0.99 |
| arrayWithProxy get 100000 times | 29.042 ms/op | 29.500 ms/op | 0.98 |
| ssz.Root.equals | 410.00 ns/op | 505.00 ns/op | 0.81 |
| byteArrayEquals | 408.00 ns/op | 490.00 ns/op | 0.83 |
| shuffle list - 16384 els | 11.215 ms/op | 11.193 ms/op | 1.00 |
| shuffle list - 250000 els | 165.09 ms/op | 169.36 ms/op | 0.97 |
| processSlot - 1 slots | 11.298 us/op | 13.208 us/op | 0.86 |
| processSlot - 32 slots | 1.6822 ms/op | 1.8552 ms/op | 0.91 |
| getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei | 396.45 us/op | 562.84 us/op | 0.70 |
| getCommitteeAssignments - req 1 vs - 250000 vc | 5.2731 ms/op | 5.5192 ms/op | 0.96 |
| getCommitteeAssignments - req 100 vs - 250000 vc | 7.3187 ms/op | 7.5594 ms/op | 0.97 |
| getCommitteeAssignments - req 1000 vs - 250000 vc | 7.7582 ms/op | 7.9836 ms/op | 0.97 |
| RootCache.getBlockRootAtSlot - 250000 vs - 7PWei | 9.0800 ns/op | 10.230 ns/op | 0.89 |
| state getBlockRootAtSlot - 250000 vs - 7PWei | 1.0130 us/op | 1.4074 us/op | 0.72 |
| computeProposers - vc 250000 | 14.940 ms/op | 18.051 ms/op | 0.83 |
| computeEpochShuffling - vc 250000 | 162.76 ms/op | 173.49 ms/op | 0.94 |
| getNextSyncCommittee - vc 250000 | 269.72 ms/op | 293.89 ms/op | 0.92 |
by benchmarkbot/action
setting this as draft
Tests work fine now