lodestar icon indicating copy to clipboard operation
lodestar copied to clipboard

Remove testcontainers dependency for web3signer test

Open dapplion opened this issue 3 years ago • 1 comments

Motivation

  • PR https://github.com/ChainSafe/lodestar/pull/4502 introduced a dev testcontainers since @dadepo was having issues with docker

I really want to start decreasing our dependencies exposure, not increasing it unless required. We have rolled our docker runners in the past and should continue to do so.

Description

  • Use adhoc docker

@dadepo I've removed the use of --network=host. If you have to run this tests locally try to understand the incompatibilities and fix them together without 3rd party libraries

dapplion avatar Sep 19 '22 12:09 dapplion

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 9ef50342c0d421e3d9f846f4017331970b203bae Previous: 8709e3a7a718e634dcc897414f7595bc97b6cd9e Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.0222 ms/op 2.2726 ms/op 0.89
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 64.627 us/op 85.034 us/op 0.76
BLS verify - blst-native 1.6398 ms/op 1.8825 ms/op 0.87
BLS verifyMultipleSignatures 3 - blst-native 3.3554 ms/op 3.8568 ms/op 0.87
BLS verifyMultipleSignatures 8 - blst-native 7.2232 ms/op 8.3030 ms/op 0.87
BLS verifyMultipleSignatures 32 - blst-native 26.180 ms/op 30.109 ms/op 0.87
BLS aggregatePubkeys 32 - blst-native 34.879 us/op 39.878 us/op 0.87
BLS aggregatePubkeys 128 - blst-native 135.23 us/op 154.58 us/op 0.87
getAttestationsForBlock 82.174 ms/op 98.900 ms/op 0.83
isKnown best case - 1 super set check 435.00 ns/op 440.00 ns/op 0.99
isKnown normal case - 2 super set checks 425.00 ns/op 419.00 ns/op 1.01
isKnown worse case - 16 super set checks 424.00 ns/op 420.00 ns/op 1.01
CheckpointStateCache - add get delete 8.9270 us/op 9.4480 us/op 0.94
validate gossip signedAggregateAndProof - struct 4.2520 ms/op 4.3875 ms/op 0.97
validate gossip attestation - struct 2.0347 ms/op 2.0799 ms/op 0.98
pickEth1Vote - no votes 2.1913 ms/op 2.9606 ms/op 0.74
pickEth1Vote - max votes 19.643 ms/op 25.381 ms/op 0.77
pickEth1Vote - Eth1Data hashTreeRoot value x2048 10.082 ms/op 11.991 ms/op 0.84
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 18.514 ms/op 23.462 ms/op 0.79
pickEth1Vote - Eth1Data fastSerialize value x2048 1.3556 ms/op 1.6052 ms/op 0.84
pickEth1Vote - Eth1Data fastSerialize tree x2048 13.362 ms/op 16.664 ms/op 0.80
bytes32 toHexString 965.00 ns/op 1.2770 us/op 0.76
bytes32 Buffer.toString(hex) 634.00 ns/op 767.00 ns/op 0.83
bytes32 Buffer.toString(hex) from Uint8Array 852.00 ns/op 1.0340 us/op 0.82
bytes32 Buffer.toString(hex) + 0x 631.00 ns/op 789.00 ns/op 0.80
Object access 1 prop 0.30800 ns/op 0.42600 ns/op 0.72
Map access 1 prop 0.25700 ns/op 0.28900 ns/op 0.89
Object get x1000 15.999 ns/op 17.974 ns/op 0.89
Map get x1000 0.89200 ns/op 0.99200 ns/op 0.90
Object set x1000 105.45 ns/op 134.97 ns/op 0.78
Map set x1000 63.959 ns/op 84.185 ns/op 0.76
Return object 10000 times 0.33380 ns/op 0.37530 ns/op 0.89
Throw Error 10000 times 5.1964 us/op 5.9733 us/op 0.87
enrSubnets - fastDeserialize 64 bits 2.3550 us/op 3.2570 us/op 0.72
enrSubnets - ssz BitVector 64 bits 661.00 ns/op 834.00 ns/op 0.79
enrSubnets - fastDeserialize 4 bits 352.00 ns/op 431.00 ns/op 0.82
enrSubnets - ssz BitVector 4 bits 669.00 ns/op 858.00 ns/op 0.78
prioritizePeers score -10:0 att 32-0.1 sync 2-0 81.743 us/op 106.93 us/op 0.76
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 117.89 us/op 142.90 us/op 0.82
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 195.46 us/op 253.84 us/op 0.77
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 341.72 us/op 392.85 us/op 0.87
prioritizePeers score 0:0 att 64-1 sync 4-1 407.04 us/op 464.72 us/op 0.88
RateTracker 1000000 limit, 1 obj count per request 180.84 ns/op 214.54 ns/op 0.84
RateTracker 1000000 limit, 2 obj count per request 141.45 ns/op 167.47 ns/op 0.84
RateTracker 1000000 limit, 4 obj count per request 122.22 ns/op 146.13 ns/op 0.84
RateTracker 1000000 limit, 8 obj count per request 112.73 ns/op 132.82 ns/op 0.85
RateTracker with prune 4.2450 us/op 5.7180 us/op 0.74
array of 16000 items push then shift 2.7287 us/op 3.2262 us/op 0.85
LinkedList of 16000 items push then shift 15.950 ns/op 18.030 ns/op 0.88
array of 16000 items push then pop 226.01 ns/op 266.67 ns/op 0.85
LinkedList of 16000 items push then pop 14.858 ns/op 17.137 ns/op 0.87
array of 24000 items push then shift 4.0358 us/op 4.6542 us/op 0.87
LinkedList of 24000 items push then shift 18.618 ns/op 21.458 ns/op 0.87
array of 24000 items push then pop 203.86 ns/op 236.47 ns/op 0.86
LinkedList of 24000 items push then pop 18.274 ns/op 18.616 ns/op 0.98
intersect bitArray bitLen 8 11.770 ns/op 11.679 ns/op 1.01
intersect array and set length 8 174.24 ns/op 199.97 ns/op 0.87
intersect bitArray bitLen 128 61.923 ns/op 72.499 ns/op 0.85
intersect array and set length 128 2.3202 us/op 2.4437 us/op 0.95
Buffer.concat 32 items 2.2380 ns/op 1.9930 ns/op 1.12
pass gossip attestations to forkchoice per slot 3.1519 ms/op 3.6377 ms/op 0.87
computeDeltas 3.4072 ms/op 3.1617 ms/op 1.08
computeProposerBoostScoreFromBalances 812.76 us/op 914.51 us/op 0.89
altair processAttestation - 250000 vs - 7PWei normalcase 3.2025 ms/op 4.4580 ms/op 0.72
altair processAttestation - 250000 vs - 7PWei worstcase 4.9510 ms/op 6.7126 ms/op 0.74
altair processAttestation - setStatus - 1/6 committees join 185.69 us/op 228.91 us/op 0.81
altair processAttestation - setStatus - 1/3 committees join 355.94 us/op 425.20 us/op 0.84
altair processAttestation - setStatus - 1/2 committees join 556.44 us/op 594.43 us/op 0.94
altair processAttestation - setStatus - 2/3 committees join 727.53 us/op 801.28 us/op 0.91
altair processAttestation - setStatus - 4/5 committees join 991.44 us/op 1.0664 ms/op 0.93
altair processAttestation - setStatus - 100% committees join 1.1714 ms/op 1.2616 ms/op 0.93
altair processBlock - 250000 vs - 7PWei normalcase 25.769 ms/op 30.163 ms/op 0.85
altair processBlock - 250000 vs - 7PWei normalcase hashState 39.630 ms/op 37.941 ms/op 1.04
altair processBlock - 250000 vs - 7PWei worstcase 75.837 ms/op 93.205 ms/op 0.81
altair processBlock - 250000 vs - 7PWei worstcase hashState 106.22 ms/op 99.096 ms/op 1.07
phase0 processBlock - 250000 vs - 7PWei normalcase 3.4381 ms/op 4.2678 ms/op 0.81
phase0 processBlock - 250000 vs - 7PWei worstcase 45.243 ms/op 48.020 ms/op 0.94
altair processEth1Data - 250000 vs - 7PWei normalcase 770.12 us/op 1.0748 ms/op 0.72
Tree 40 250000 create 754.12 ms/op 1.1181 s/op 0.67
Tree 40 250000 get(125000) 253.01 ns/op 294.40 ns/op 0.86
Tree 40 250000 set(125000) 2.4956 us/op 3.5923 us/op 0.69
Tree 40 250000 toArray() 30.045 ms/op 33.267 ms/op 0.90
Tree 40 250000 iterate all - toArray() + loop 30.566 ms/op 32.901 ms/op 0.93
Tree 40 250000 iterate all - get(i) 100.42 ms/op 112.08 ms/op 0.90
MutableVector 250000 create 16.145 ms/op 16.934 ms/op 0.95
MutableVector 250000 get(125000) 13.110 ns/op 14.920 ns/op 0.88
MutableVector 250000 set(125000) 642.38 ns/op 907.45 ns/op 0.71
MutableVector 250000 toArray() 7.2206 ms/op 7.9888 ms/op 0.90
MutableVector 250000 iterate all - toArray() + loop 7.3013 ms/op 8.1623 ms/op 0.89
MutableVector 250000 iterate all - get(i) 3.4414 ms/op 3.4788 ms/op 0.99
Array 250000 create 6.8358 ms/op 7.3719 ms/op 0.93
Array 250000 clone - spread 4.0637 ms/op 5.7128 ms/op 0.71
Array 250000 get(125000) 1.7400 ns/op 1.7920 ns/op 0.97
Array 250000 set(125000) 1.7370 ns/op 1.7590 ns/op 0.99
Array 250000 iterate all - loop 170.39 us/op 170.56 us/op 1.00
effectiveBalanceIncrements clone Uint8Array 300000 84.277 us/op 93.430 us/op 0.90
effectiveBalanceIncrements clone MutableVector 300000 706.00 ns/op 1.3360 us/op 0.53
effectiveBalanceIncrements rw all Uint8Array 300000 252.38 us/op 254.72 us/op 0.99
effectiveBalanceIncrements rw all MutableVector 300000 156.87 ms/op 240.64 ms/op 0.65
phase0 afterProcessEpoch - 250000 vs - 7PWei 197.08 ms/op 202.99 ms/op 0.97
phase0 beforeProcessEpoch - 250000 vs - 7PWei 71.213 ms/op 77.585 ms/op 0.92
altair processEpoch - mainnet_e81889 554.07 ms/op 608.57 ms/op 0.91
mainnet_e81889 - altair beforeProcessEpoch 140.93 ms/op 150.38 ms/op 0.94
mainnet_e81889 - altair processJustificationAndFinalization 23.247 us/op 25.842 us/op 0.90
mainnet_e81889 - altair processInactivityUpdates 10.422 ms/op 10.876 ms/op 0.96
mainnet_e81889 - altair processRewardsAndPenalties 92.069 ms/op 97.267 ms/op 0.95
mainnet_e81889 - altair processRegistryUpdates 4.8210 us/op 4.5440 us/op 1.06
mainnet_e81889 - altair processSlashings 913.00 ns/op 1.0210 us/op 0.89
mainnet_e81889 - altair processEth1DataReset 984.00 ns/op 1.0870 us/op 0.91
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.4372 ms/op 2.3044 ms/op 1.06
mainnet_e81889 - altair processSlashingsReset 6.7470 us/op 7.0060 us/op 0.96
mainnet_e81889 - altair processRandaoMixesReset 7.3830 us/op 7.8360 us/op 0.94
mainnet_e81889 - altair processHistoricalRootsUpdate 830.00 ns/op 993.00 ns/op 0.84
mainnet_e81889 - altair processParticipationFlagUpdates 3.5930 us/op 4.2250 us/op 0.85
mainnet_e81889 - altair processSyncCommitteeUpdates 860.00 ns/op 861.00 ns/op 1.00
mainnet_e81889 - altair afterProcessEpoch 174.50 ms/op 200.51 ms/op 0.87
phase0 processEpoch - mainnet_e58758 501.60 ms/op 549.18 ms/op 0.91
mainnet_e58758 - phase0 beforeProcessEpoch 222.04 ms/op 247.21 ms/op 0.90
mainnet_e58758 - phase0 processJustificationAndFinalization 26.124 us/op 26.690 us/op 0.98
mainnet_e58758 - phase0 processRewardsAndPenalties 134.02 ms/op 146.34 ms/op 0.92
mainnet_e58758 - phase0 processRegistryUpdates 12.292 us/op 12.111 us/op 1.01
mainnet_e58758 - phase0 processSlashings 1.0840 us/op 903.00 ns/op 1.20
mainnet_e58758 - phase0 processEth1DataReset 1.0710 us/op 952.00 ns/op 1.13
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.1569 ms/op 2.1789 ms/op 0.99
mainnet_e58758 - phase0 processSlashingsReset 7.1740 us/op 6.5810 us/op 1.09
mainnet_e58758 - phase0 processRandaoMixesReset 8.5640 us/op 7.6350 us/op 1.12
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.4840 us/op 1.0740 us/op 1.38
mainnet_e58758 - phase0 processParticipationRecordUpdates 5.6790 us/op 6.5570 us/op 0.87
mainnet_e58758 - phase0 afterProcessEpoch 142.14 ms/op 162.71 ms/op 0.87
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.7685 ms/op 2.6840 ms/op 1.03
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.4136 ms/op 3.5223 ms/op 0.97
altair processInactivityUpdates - 250000 normalcase 47.456 ms/op 44.367 ms/op 1.07
altair processInactivityUpdates - 250000 worstcase 38.274 ms/op 54.059 ms/op 0.71
phase0 processRegistryUpdates - 250000 normalcase 10.520 us/op 10.129 us/op 1.04
phase0 processRegistryUpdates - 250000 badcase_full_deposits 480.08 us/op 440.28 us/op 1.09
phase0 processRegistryUpdates - 250000 worstcase 0.5 207.71 ms/op 227.31 ms/op 0.91
altair processRewardsAndPenalties - 250000 normalcase 78.602 ms/op 136.34 ms/op 0.58
altair processRewardsAndPenalties - 250000 worstcase 80.028 ms/op 90.286 ms/op 0.89
phase0 getAttestationDeltas - 250000 normalcase 11.628 ms/op 16.316 ms/op 0.71
phase0 getAttestationDeltas - 250000 worstcase 12.177 ms/op 16.513 ms/op 0.74
phase0 processSlashings - 250000 worstcase 5.3292 ms/op 5.6272 ms/op 0.95
altair processSyncCommitteeUpdates - 250000 269.76 ms/op 300.73 ms/op 0.90
BeaconState.hashTreeRoot - No change 499.00 ns/op 480.00 ns/op 1.04
BeaconState.hashTreeRoot - 1 full validator 57.373 us/op 63.186 us/op 0.91
BeaconState.hashTreeRoot - 32 full validator 572.06 us/op 784.36 us/op 0.73
BeaconState.hashTreeRoot - 512 full validator 7.2641 ms/op 6.6026 ms/op 1.10
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 69.855 us/op 80.821 us/op 0.86
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.1557 ms/op 1.2353 ms/op 0.94
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 14.635 ms/op 17.137 ms/op 0.85
BeaconState.hashTreeRoot - 1 balances 57.768 us/op 53.144 us/op 1.09
BeaconState.hashTreeRoot - 32 balances 570.57 us/op 580.55 us/op 0.98
BeaconState.hashTreeRoot - 512 balances 5.4472 ms/op 6.3355 ms/op 0.86
BeaconState.hashTreeRoot - 250000 balances 78.536 ms/op 96.927 ms/op 0.81
aggregationBits - 2048 els - zipIndexesInBitList 34.459 us/op 43.217 us/op 0.80
regular array get 100000 times 67.419 us/op 68.193 us/op 0.99
wrappedArray get 100000 times 67.401 us/op 68.112 us/op 0.99
arrayWithProxy get 100000 times 29.042 ms/op 29.500 ms/op 0.98
ssz.Root.equals 410.00 ns/op 505.00 ns/op 0.81
byteArrayEquals 408.00 ns/op 490.00 ns/op 0.83
shuffle list - 16384 els 11.215 ms/op 11.193 ms/op 1.00
shuffle list - 250000 els 165.09 ms/op 169.36 ms/op 0.97
processSlot - 1 slots 11.298 us/op 13.208 us/op 0.86
processSlot - 32 slots 1.6822 ms/op 1.8552 ms/op 0.91
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 396.45 us/op 562.84 us/op 0.70
getCommitteeAssignments - req 1 vs - 250000 vc 5.2731 ms/op 5.5192 ms/op 0.96
getCommitteeAssignments - req 100 vs - 250000 vc 7.3187 ms/op 7.5594 ms/op 0.97
getCommitteeAssignments - req 1000 vs - 250000 vc 7.7582 ms/op 7.9836 ms/op 0.97
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 9.0800 ns/op 10.230 ns/op 0.89
state getBlockRootAtSlot - 250000 vs - 7PWei 1.0130 us/op 1.4074 us/op 0.72
computeProposers - vc 250000 14.940 ms/op 18.051 ms/op 0.83
computeEpochShuffling - vc 250000 162.76 ms/op 173.49 ms/op 0.94
getNextSyncCommittee - vc 250000 269.72 ms/op 293.89 ms/op 0.92

by benchmarkbot/action

github-actions[bot] avatar Sep 19 '22 12:09 github-actions[bot]

setting this as draft

wemeetagain avatar Oct 30 '22 18:10 wemeetagain

Tests work fine now

dapplion avatar May 29 '23 11:05 dapplion