aptos-core icon indicating copy to clipboard operation
aptos-core copied to clipboard

[test on forge] mempool current buckets

Open igor-aptos opened this issue 3 years ago • 22 comments

Description

Test Plan


This change is Reviewable

igor-aptos avatar Oct 11 '22 01:10 igor-aptos

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> eb62a9d6944ddf38bdaa18a81dd4942fc374bca7

github-actions[bot] avatar Oct 11 '22 03:10 github-actions[bot]

Forge is running suite land_blocking on eb62a9d6944ddf38bdaa18a81dd4942fc374bca7

github-actions[bot] avatar Oct 11 '22 03:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> eb62a9d6944ddf38bdaa18a81dd4942fc374bca7

Forge test runner terminated:
Trailing Log Lines:
thread 'main' panicked at 'explicit panic', testsuite/forge-cli/src/main.rs:191:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Debugging output:

github-actions[bot] avatar Oct 11 '22 03:10 github-actions[bot]

:x: Forge suite land_blocking failure on eb62a9d6944ddf38bdaa18a81dd4942fc374bca7

performance benchmark with full nodes : 5984 TPS, 621 ms latency, 44400 ms p99 latency,(!) expired 1715962 out of 4271500 txns
Test Failed: check for success

Caused by:
    TPS requirement failed. Average TPS 5984, minimum TPS requirement 6500
Trailing Log Lines:

Caused by:
    TPS requirement failed. Average TPS 5984, minimum TPS requirement 6500
Test Statistics: 
performance benchmark with full nodes : 5984 TPS, 621 ms latency, 44400 ms p99 latency,(!) expired 1715962 out of 4271500 txns
Test Failed: check for success

Caused by:
    TPS requirement failed. Average TPS 5984, minimum TPS requirement 6500


Swarm logs can be found here: See fgi output for more information.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-e2e-pr-4945-1665457686-eb62a9d6944ddf38bdaa18a81dd4942fc3","timestamp":"2022-10-11T03:25:37.659417Z","message":"Deleting namespace forge-e2e-pr-4945: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-e2e-pr-4945-1665457686-eb62a9d6944ddf38bdaa18a81dd4942fc3","timestamp":"2022-10-11T03:25:37.659451Z","message":"aptos-node resources for Forge removed in namespace: forge-e2e-pr-4945"}

failures:
    performance benchmark with full nodes

test result: FAILED. 0 passed; 1 failed; 0 filtered out

Failed to run tests:
Tests Failed
Error: Tests Failed
Debugging output:
NAME                                    READY   STATUS      RESTARTS   AGE
aptos-node-2-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-2-validator-0                1/1     Running     0          14m
aptos-node-3-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-3-validator-0                1/1     Running     0          14m
aptos-node-4-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-4-validator-0                1/1     Running     0          14m
aptos-node-5-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-5-validator-0                1/1     Running     0          14m
aptos-node-6-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-6-validator-0                1/1     Running     0          14m
aptos-node-7-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-7-validator-0                1/1     Running     0          14m
aptos-node-8-validator-0                1/1     Running     0          14m
aptos-node-9-validator-0                1/1     Running     0          14m
genesis-aptos-genesis-eforge237-872t5   0/1     Completed   0          16m

github-actions[bot] avatar Oct 11 '22 03:10 github-actions[bot]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 21eb308af50b2f2664f324aa48e1b8008fb80b68

github-actions[bot] avatar Oct 11 '22 17:10 github-actions[bot]

Forge is running suite land_blocking on 21eb308af50b2f2664f324aa48e1b8008fb80b68

github-actions[bot] avatar Oct 11 '22 17:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> 21eb308af50b2f2664f324aa48e1b8008fb80b68

Forge test runner terminated:
Trailing Log Lines:
thread 'main' panicked at 'explicit panic', testsuite/forge-cli/src/main.rs:191:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Debugging output:

github-actions[bot] avatar Oct 11 '22 17:10 github-actions[bot]

:x: Forge suite land_blocking failure on 21eb308af50b2f2664f324aa48e1b8008fb80b68

Test Failed: start emitter job

Caused by:
    Failed to mint accounts: Account ddadbccf6a4614788a8dc9e0bd9ecea96f575c1a0c2db08facbc0190154fe6e3 couldn't mint
    
    Caused by:
        Failed to wait for transactions: Unknown(Transaction expired, it is guaranteed it will not be committed on chain.)
Trailing Log Lines:
        Failed to wait for transactions: Unknown(Transaction expired, it is guaranteed it will not be committed on chain.)
Test Statistics: 
Test Failed: start emitter job

Caused by:
    Failed to mint accounts: Account ddadbccf6a4614788a8dc9e0bd9ecea96f575c1a0c2db08facbc0190154fe6e3 couldn't mint
    
    Caused by:
        Failed to wait for transactions: Unknown(Transaction expired, it is guaranteed it will not be committed on chain.)


Swarm logs can be found here: See fgi output for more information.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-e2e-pr-4945-1665509427-21eb308af50b2f2664f324aa48e1b8008f","timestamp":"2022-10-11T17:37:52.844656Z","message":"Deleting namespace forge-e2e-pr-4945: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-e2e-pr-4945-1665509427-21eb308af50b2f2664f324aa48e1b8008f","timestamp":"2022-10-11T17:37:52.844697Z","message":"aptos-node resources for Forge removed in namespace: forge-e2e-pr-4945"}

failures:
    performance benchmark with full nodes

test result: FAILED. 0 passed; 1 failed; 0 filtered out
Failed to run tests:
Tests Failed

Error: Tests Failed
Debugging output:
NAME                                    READY   STATUS      RESTARTS   AGE
aptos-node-0-fullnode-eforge239-0       1/1     Running     0          4m8s
aptos-node-0-validator-0                1/1     Running     0          4m8s
aptos-node-1-fullnode-eforge239-0       1/1     Running     0          4m8s
aptos-node-1-validator-0                1/1     Running     0          4m8s
aptos-node-2-fullnode-eforge239-0       1/1     Running     0          4m8s
aptos-node-2-validator-0                1/1     Running     0          4m8s
aptos-node-3-fullnode-eforge239-0       1/1     Running     0          4m8s
aptos-node-3-validator-0                1/1     Running     0          4m8s
aptos-node-4-fullnode-eforge239-0       1/1     Running     0          4m8s
aptos-node-4-validator-0                1/1     Running     0          4m8s
aptos-node-5-fullnode-eforge239-0       1/1     Running     0          4m8s
aptos-node-5-validator-0                1/1     Running     0          4m8s
aptos-node-6-fullnode-eforge239-0       1/1     Running     0          4m8s
aptos-node-6-validator-0                1/1     Running     0          4m8s
aptos-node-7-fullnode-eforge239-0       1/1     Running     0          4m8s
aptos-node-7-validator-0                1/1     Running     0          4m8s
aptos-node-8-validator-0                1/1     Running     0          4m8s
aptos-node-9-validator-0                1/1     Running     0          4m8s
genesis-aptos-genesis-eforge239-kq6sm   0/1     Completed   0          6m45s

github-actions[bot] avatar Oct 11 '22 17:10 github-actions[bot]

Forge is running suite land_blocking on 21eb308af50b2f2664f324aa48e1b8008fb80b68

github-actions[bot] avatar Oct 11 '22 17:10 github-actions[bot]

:white_check_mark: Forge suite land_blocking success on 21eb308af50b2f2664f324aa48e1b8008fb80b68

performance benchmark with full nodes : 6637 TPS, 1024 ms latency, 67100 ms p99 latency,(!) expired 2283040 out of 5117140 txns
Test Ok

github-actions[bot] avatar Oct 11 '22 18:10 github-actions[bot]

Forge is running suite land_blocking on a63cc72d959edc91246e671882e9cf7da3175db7

github-actions[bot] avatar Oct 11 '22 19:10 github-actions[bot]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> a63cc72d959edc91246e671882e9cf7da3175db7

github-actions[bot] avatar Oct 11 '22 19:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> a63cc72d959edc91246e671882e9cf7da3175db7

Forge test runner terminated:
Trailing Log Lines:
thread 'main' panicked at 'explicit panic', testsuite/forge-cli/src/main.rs:191:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Debugging output:

github-actions[bot] avatar Oct 11 '22 19:10 github-actions[bot]

:x: Forge suite land_blocking failure on a63cc72d959edc91246e671882e9cf7da3175db7

performance benchmark with full nodes : 6335 TPS, 1098 ms latency, 67300 ms p99 latency,(!) expired 2416752 out of 5122120 txns
Test Failed: check for success

Caused by:
    TPS requirement failed. Average TPS 6335, minimum TPS requirement 6500
Trailing Log Lines:

Caused by:
    TPS requirement failed. Average TPS 6335, minimum TPS requirement 6500
Test Statistics: 
performance benchmark with full nodes : 6335 TPS, 1098 ms latency, 67300 ms p99 latency,(!) expired 2416752 out of 5122120 txns
Test Failed: check for success

Caused by:
    TPS requirement failed. Average TPS 6335, minimum TPS requirement 6500


Swarm logs can be found here: See fgi output for more information.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-e2e-pr-4945-1665515045-a63cc72d959edc91246e671882e9cf7da3","timestamp":"2022-10-11T19:20:54.628210Z","message":"Deleting namespace forge-e2e-pr-4945: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-e2e-pr-4945-1665515045-a63cc72d959edc91246e671882e9cf7da3","timestamp":"2022-10-11T19:20:54.628235Z","message":"aptos-node resources for Forge removed in namespace: forge-e2e-pr-4945"}

failures:
    performance benchmark with full nodes

Failed to run tests:
Tests Failed
test result: FAILED. 0 passed; 1 failed; 0 filtered out

Error: Tests Failed
Debugging output:
NAME                                    READY   STATUS      RESTARTS   AGE
aptos-node-0-fullnode-eforge161-0       1/1     Running     0          13m
aptos-node-0-validator-0                1/1     Running     0          13m
aptos-node-1-fullnode-eforge161-0       1/1     Running     0          13m
aptos-node-1-validator-0                1/1     Running     0          13m
aptos-node-2-fullnode-eforge161-0       1/1     Running     0          13m
aptos-node-2-validator-0                1/1     Running     0          13m
aptos-node-3-fullnode-eforge161-0       1/1     Running     0          13m
aptos-node-3-validator-0                1/1     Running     0          13m
aptos-node-4-fullnode-eforge161-0       1/1     Running     0          13m
aptos-node-4-validator-0                1/1     Running     0          13m
aptos-node-5-fullnode-eforge161-0       1/1     Running     0          13m
aptos-node-5-validator-0                1/1     Running     0          13m
aptos-node-6-fullnode-eforge161-0       1/1     Running     0          13m
aptos-node-6-validator-0                1/1     Running     0          13m
aptos-node-7-fullnode-eforge161-0       1/1     Running     0          13m
aptos-node-7-validator-0                1/1     Running     0          13m
aptos-node-8-validator-0                1/1     Running     0          13m
aptos-node-9-validator-0                1/1     Running     0          13m
genesis-aptos-genesis-eforge161-jzzxj   0/1     Completed   0          16m

github-actions[bot] avatar Oct 11 '22 19:10 github-actions[bot]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 7d51e51b85eb0b5433d0288b82ec5c45d445ff0c

github-actions[bot] avatar Oct 12 '22 19:10 github-actions[bot]

Forge is running suite land_blocking on 7d51e51b85eb0b5433d0288b82ec5c45d445ff0c

github-actions[bot] avatar Oct 12 '22 19:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> 7d51e51b85eb0b5433d0288b82ec5c45d445ff0c

Forge test runner terminated:
Trailing Log Lines:
thread 'main' panicked at 'explicit panic', testsuite/forge-cli/src/main.rs:191:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Debugging output:

github-actions[bot] avatar Oct 12 '22 19:10 github-actions[bot]

:x: Forge suite land_blocking failure on 7d51e51b85eb0b5433d0288b82ec5c45d445ff0c

performance benchmark with full nodes : 4952 TPS, 224 ms latency, 62000 ms p99 latency,(!) expired 2975960 out of 5090860 txns
Test Failed: check for success

Caused by:
    TPS requirement failed. Average TPS 4952, minimum TPS requirement 6500
Trailing Log Lines:

Caused by:
    TPS requirement failed. Average TPS 4952, minimum TPS requirement 6500
Test Statistics: 
performance benchmark with full nodes : 4952 TPS, 224 ms latency, 62000 ms p99 latency,(!) expired 2975960 out of 5090860 txns
Test Failed: check for success

Caused by:
    TPS requirement failed. Average TPS 4952, minimum TPS requirement 6500


Swarm logs can be found here: See fgi output for more information.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-e2e-pr-4945-1665603777-7d51e51b85eb0b5433d0288b82ec5c45d4","timestamp":"2022-10-12T20:01:30.306036Z","message":"Deleting namespace forge-e2e-pr-4945: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-e2e-pr-4945-1665603777-7d51e51b85eb0b5433d0288b82ec5c45d4","timestamp":"2022-10-12T20:01:30.306070Z","message":"aptos-node resources for Forge removed in namespace: forge-e2e-pr-4945"}

failures:
    performance benchmark with full nodes

test result: FAILED. 0 passed; 1 failed; 0 filtered out

Failed to run tests:
Tests Failed
Error: Tests Failed
Debugging output:
NAME                                    READY   STATUS      RESTARTS   AGE
aptos-node-0-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-0-validator-0                1/1     Running     0          14m
aptos-node-1-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-1-validator-0                1/1     Running     0          14m
aptos-node-2-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-2-validator-0                1/1     Running     0          14m
aptos-node-3-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-3-validator-0                1/1     Running     0          14m
aptos-node-4-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-4-validator-0                1/1     Running     0          14m
aptos-node-5-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-5-validator-0                1/1     Running     0          14m
aptos-node-6-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-6-validator-0                1/1     Running     0          14m
aptos-node-7-fullnode-eforge237-0       1/1     Running     0          14m
aptos-node-7-validator-0                1/1     Running     0          14m
aptos-node-8-validator-0                1/1     Running     0          14m
aptos-node-9-validator-0                1/1     Running     0          14m
genesis-aptos-genesis-eforge237-x4bjw   0/1     Completed   0          17m

github-actions[bot] avatar Oct 12 '22 20:10 github-actions[bot]

Forge is running suite land_blocking on a4cf1a9a2d4f75c5752a08d1272cc8a7c65f3532

github-actions[bot] avatar Oct 14 '22 23:10 github-actions[bot]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> a4cf1a9a2d4f75c5752a08d1272cc8a7c65f3532

github-actions[bot] avatar Oct 14 '22 23:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> a4cf1a9a2d4f75c5752a08d1272cc8a7c65f3532

Forge test runner terminated:
Trailing Log Lines:
thread 'main' panicked at 'explicit panic', testsuite/forge-cli/src/main.rs:191:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Debugging output:

github-actions[bot] avatar Oct 14 '22 23:10 github-actions[bot]

:x: Forge suite land_blocking failure on a4cf1a9a2d4f75c5752a08d1272cc8a7c65f3532

performance benchmark with full nodes : 5005 TPS, 230 ms latency, 61900 ms p99 latency,(!) expired 2950646 out of 5088040 txns
Test Failed: check for success

Caused by:
    TPS requirement failed. Average TPS 5005, minimum TPS requirement 6500
Trailing Log Lines:

Caused by:
    TPS requirement failed. Average TPS 5005, minimum TPS requirement 6500
Test Statistics: 
performance benchmark with full nodes : 5005 TPS, 230 ms latency, 61900 ms p99 latency,(!) expired 2950646 out of 5088040 txns
Test Failed: check for success

Caused by:
    TPS requirement failed. Average TPS 5005, minimum TPS requirement 6500


Swarm logs can be found here: See fgi output for more information.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-e2e-pr-4945-1665790282-a4cf1a9a2d4f75c5752a08d1272cc8a7c6","timestamp":"2022-10-14T23:49:30.130066Z","message":"Deleting namespace forge-e2e-pr-4945: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-e2e-pr-4945-1665790282-a4cf1a9a2d4f75c5752a08d1272cc8a7c6","timestamp":"2022-10-14T23:49:30.130095Z","message":"aptos-node resources for Forge removed in namespace: forge-e2e-pr-4945"}

failures:
    performance benchmark with full nodes

test result: FAILED. 0 passed; 1 failed; 0 filtered out

Failed to run tests:
Tests Failed
Error: Tests Failed
Debugging output:
NAME                                    READY   STATUS      RESTARTS   AGE
aptos-node-0-fullnode-eforge189-0       1/1     Running     0          14m
aptos-node-0-validator-0                1/1     Running     0          14m
aptos-node-1-fullnode-eforge189-0       1/1     Running     0          14m
aptos-node-1-validator-0                1/1     Running     0          14m
aptos-node-2-fullnode-eforge189-0       1/1     Running     0          14m
aptos-node-2-validator-0                1/1     Running     0          14m
aptos-node-3-fullnode-eforge189-0       1/1     Running     0          14m
aptos-node-3-validator-0                1/1     Running     0          14m
aptos-node-4-fullnode-eforge189-0       1/1     Running     0          14m
aptos-node-4-validator-0                1/1     Running     0          14m
aptos-node-5-fullnode-eforge189-0       1/1     Running     0          14m
aptos-node-5-validator-0                1/1     Running     0          14m
aptos-node-6-fullnode-eforge189-0       1/1     Running     0          14m
aptos-node-6-validator-0                1/1     Running     0          14m
aptos-node-7-fullnode-eforge189-0       1/1     Running     0          14m
aptos-node-7-validator-0                1/1     Running     0          14m
aptos-node-8-validator-0                1/1     Running     0          14m
aptos-node-9-validator-0                1/1     Running     0          14m
genesis-aptos-genesis-eforge189-cwpfl   0/1     Completed   0          17m

github-actions[bot] avatar Oct 14 '22 23:10 github-actions[bot]

stale PR?

davidiw avatar Oct 28 '22 16:10 davidiw