aptos-core icon indicating copy to clipboard operation
aptos-core copied to clipboard

[forge] Set random-seed parameter for key generation

Open zcchahaha opened this issue 2 years ago • 2 comments

Description

  • Add a seed parameter for keys generation so that the keys generation will be deterministic and can be accessed from forge tests.
  • The i th validator will use the passed in paremeter + i as seed to generate keys.

Test Plan

Apply the label CICD:run-e2e-tests


This change is Reviewable

zcchahaha avatar Sep 09 '22 19:09 zcchahaha

Forge is running suite land_blocking on cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge is running suite compat on testnet ==> cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

:x: Forge suite land_blocking failure on cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge test runner terminated:
Trailing Log Lines:
             at /aptos/crates/aptos/src/main.rs:20:5
  33: core::ops::function::FnOnce::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-e2e-pr-4042-1662755343-cfd7c49a63bd8950f1e0aa5de4911c578a","timestamp":"2022-09-09T20:42:53.591703Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: Some(4), start_time: Some(Time(2022-09-09T20:32:59Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:260"},"thread_name":"main","hostname":"forge-e2e-pr-4042-1662755343-cfd7c49a63bd8950f1e0aa5de4911c578a","timestamp":"2022-09-09T20:42:53.613315Z","message":"Deleting namespace forge-e2e-pr-4042: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:368"},"thread_name":"main","hostname":"forge-e2e-pr-4042-1662755343-cfd7c49a63bd8950f1e0aa5de4911c578a","timestamp":"2022-09-09T20:42:53.613349Z","message":"aptos-node resources for Forge removed in namespace: forge-e2e-pr-4042"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

Forge is running suite land_blocking on cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge is running suite compat on testnet ==> cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

:x: Forge suite compat failure on testnet ==> cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge test runner terminated:
Trailing Log Lines:
             at /aptos/crates/aptos/src/main.rs:20:5
  33: core::ops::function::FnOnce::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4042-1662760842-testnet","timestamp":"2022-09-09T22:11:28.577354Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: Some(5), start_time: Some(Time(2022-09-09T22:01:34Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:260"},"thread_name":"main","hostname":"forge-compat-pr-4042-1662760842-testnet","timestamp":"2022-09-09T22:11:28.597963Z","message":"Deleting namespace forge-compat-pr-4042: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:368"},"thread_name":"main","hostname":"forge-compat-pr-4042-1662760842-testnet","timestamp":"2022-09-09T22:11:28.597993Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4042"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

Forge is running suite land_blocking on 2483246aefbfb5e4fd5b9e6096a985f301547ffe

Forge is running suite compat on testnet ==> 2483246aefbfb5e4fd5b9e6096a985f301547ffe

:white_check_mark: Forge suite land_blocking success on 2483246aefbfb5e4fd5b9e6096a985f301547ffe

performance benchmark with full nodes : 7482 TPS, 3961 ms latency, 7200 ms p99 latency,no expired txns
Test Ok

Forge is running suite land_blocking on ed773159f49f6c9dc05f632ac1df2017b851dea4

Forge is running suite compat on testnet ==> ed773159f49f6c9dc05f632ac1df2017b851dea4

:white_check_mark: Forge suite compat success on testnet ==> ed773159f49f6c9dc05f632ac1df2017b851dea4

Compatibility test results for testnet ==> ed773159f49f6c9dc05f632ac1df2017b851dea4 (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : 6517 TPS, 4197 ms latency, 6700 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: ed773159f49f6c9dc05f632ac1df2017b851dea4
compatibility::simple-validator-upgrade::single-validator-upgrade : 5126 TPS, 5130 ms latency, 7200 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: ed773159f49f6c9dc05f632ac1df2017b851dea4
compatibility::simple-validator-upgrade::half-validator-upgrade : 5045 TPS, 5308 ms latency, 7900 ms p99 latency,no expired txns
4. upgrading second batch to new version: ed773159f49f6c9dc05f632ac1df2017b851dea4
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6903 TPS, 4001 ms latency, 6900 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet ==> ed773159f49f6c9dc05f632ac1df2017b851dea4 passed
Test Ok

Forge is running suite compat on testnet ==> c029334228a7e008509cf5fad9fd873020f26be8

Forge is running suite land_blocking on c029334228a7e008509cf5fad9fd873020f26be8

:white_check_mark: Forge suite compat success on testnet ==> c029334228a7e008509cf5fad9fd873020f26be8

Compatibility test results for testnet ==> c029334228a7e008509cf5fad9fd873020f26be8 (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : 7527 TPS, 4890 ms latency, 6900 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: c029334228a7e008509cf5fad9fd873020f26be8
compatibility::simple-validator-upgrade::single-validator-upgrade : 5462 TPS, 6602 ms latency, 8300 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: c029334228a7e008509cf5fad9fd873020f26be8
compatibility::simple-validator-upgrade::half-validator-upgrade : 5372 TPS, 7162 ms latency, 11100 ms p99 latency,no expired txns
4. upgrading second batch to new version: c029334228a7e008509cf5fad9fd873020f26be8
compatibility::simple-validator-upgrade::rest-validator-upgrade : 7887 TPS, 4763 ms latency, 7400 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet ==> c029334228a7e008509cf5fad9fd873020f26be8 passed
Test Ok

Forge is running suite land_blocking on 45bd3dd2dcfd521d01718a2d4a87a0876401833a

Forge is running suite compat on testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a

:white_check_mark: Forge suite land_blocking success on 45bd3dd2dcfd521d01718a2d4a87a0876401833a

performance benchmark with full nodes : 7809 TPS, 5081 ms latency, 8400 ms p99 latency,no expired txns
Test Ok

Forge is running suite compat on testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a

Forge is running suite land_blocking on 45bd3dd2dcfd521d01718a2d4a87a0876401833a

:white_check_mark: Forge suite land_blocking success on 45bd3dd2dcfd521d01718a2d4a87a0876401833a

performance benchmark with full nodes : 7969 TPS, 4979 ms latency, 6900 ms p99 latency,no expired txns
Test Ok

Forge is running suite compat on testnet ==> 3e048f6b9a300f681b1f14f3ff515e9de209750d

Forge is running suite land_blocking on 3e048f6b9a300f681b1f14f3ff515e9de209750d

:white_check_mark: Forge suite compat success on testnet ==> 3e048f6b9a300f681b1f14f3ff515e9de209750d

Compatibility test results for testnet ==> 3e048f6b9a300f681b1f14f3ff515e9de209750d (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : 8423 TPS, 4362 ms latency, 6300 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 3e048f6b9a300f681b1f14f3ff515e9de209750d
compatibility::simple-validator-upgrade::single-validator-upgrade : 5648 TPS, 6594 ms latency, 8900 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 3e048f6b9a300f681b1f14f3ff515e9de209750d
compatibility::simple-validator-upgrade::half-validator-upgrade : 5248 TPS, 7003 ms latency, 9400 ms p99 latency,no expired txns
4. upgrading second batch to new version: 3e048f6b9a300f681b1f14f3ff515e9de209750d
compatibility::simple-validator-upgrade::rest-validator-upgrade : 8068 TPS, 4540 ms latency, 7600 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet ==> 3e048f6b9a300f681b1f14f3ff515e9de209750d passed
Test Ok

github-actions[bot] avatar Sep 14 '22 18:09 github-actions[bot]

Forge is running suite land_blocking on cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge is running suite compat on testnet ==> cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

:x: Forge suite land_blocking failure on cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge test runner terminated:
Trailing Log Lines:
             at /aptos/crates/aptos/src/main.rs:20:5
  33: core::ops::function::FnOnce::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-e2e-pr-4042-1662755343-cfd7c49a63bd8950f1e0aa5de4911c578a","timestamp":"2022-09-09T20:42:53.591703Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: Some(4), start_time: Some(Time(2022-09-09T20:32:59Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:260"},"thread_name":"main","hostname":"forge-e2e-pr-4042-1662755343-cfd7c49a63bd8950f1e0aa5de4911c578a","timestamp":"2022-09-09T20:42:53.613315Z","message":"Deleting namespace forge-e2e-pr-4042: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:368"},"thread_name":"main","hostname":"forge-e2e-pr-4042-1662755343-cfd7c49a63bd8950f1e0aa5de4911c578a","timestamp":"2022-09-09T20:42:53.613349Z","message":"aptos-node resources for Forge removed in namespace: forge-e2e-pr-4042"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

Forge is running suite land_blocking on cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge is running suite compat on testnet ==> cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

:x: Forge suite compat failure on testnet ==> cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge test runner terminated:
Trailing Log Lines:
             at /aptos/crates/aptos/src/main.rs:20:5
  33: core::ops::function::FnOnce::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4042-1662760842-testnet","timestamp":"2022-09-09T22:11:28.577354Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: Some(5), start_time: Some(Time(2022-09-09T22:01:34Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:260"},"thread_name":"main","hostname":"forge-compat-pr-4042-1662760842-testnet","timestamp":"2022-09-09T22:11:28.597963Z","message":"Deleting namespace forge-compat-pr-4042: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:368"},"thread_name":"main","hostname":"forge-compat-pr-4042-1662760842-testnet","timestamp":"2022-09-09T22:11:28.597993Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4042"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

Forge is running suite land_blocking on 2483246aefbfb5e4fd5b9e6096a985f301547ffe

Forge is running suite compat on testnet ==> 2483246aefbfb5e4fd5b9e6096a985f301547ffe

:white_check_mark: Forge suite land_blocking success on 2483246aefbfb5e4fd5b9e6096a985f301547ffe

performance benchmark with full nodes : 7482 TPS, 3961 ms latency, 7200 ms p99 latency,no expired txns
Test Ok

Forge is running suite land_blocking on ed773159f49f6c9dc05f632ac1df2017b851dea4

Forge is running suite compat on testnet ==> ed773159f49f6c9dc05f632ac1df2017b851dea4

:white_check_mark: Forge suite compat success on testnet ==> ed773159f49f6c9dc05f632ac1df2017b851dea4

Compatibility test results for testnet ==> ed773159f49f6c9dc05f632ac1df2017b851dea4 (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : 6517 TPS, 4197 ms latency, 6700 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: ed773159f49f6c9dc05f632ac1df2017b851dea4
compatibility::simple-validator-upgrade::single-validator-upgrade : 5126 TPS, 5130 ms latency, 7200 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: ed773159f49f6c9dc05f632ac1df2017b851dea4
compatibility::simple-validator-upgrade::half-validator-upgrade : 5045 TPS, 5308 ms latency, 7900 ms p99 latency,no expired txns
4. upgrading second batch to new version: ed773159f49f6c9dc05f632ac1df2017b851dea4
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6903 TPS, 4001 ms latency, 6900 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet ==> ed773159f49f6c9dc05f632ac1df2017b851dea4 passed
Test Ok

Forge is running suite compat on testnet ==> c029334228a7e008509cf5fad9fd873020f26be8

Forge is running suite land_blocking on c029334228a7e008509cf5fad9fd873020f26be8

:white_check_mark: Forge suite compat success on testnet ==> c029334228a7e008509cf5fad9fd873020f26be8

Compatibility test results for testnet ==> c029334228a7e008509cf5fad9fd873020f26be8 (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : 7527 TPS, 4890 ms latency, 6900 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: c029334228a7e008509cf5fad9fd873020f26be8
compatibility::simple-validator-upgrade::single-validator-upgrade : 5462 TPS, 6602 ms latency, 8300 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: c029334228a7e008509cf5fad9fd873020f26be8
compatibility::simple-validator-upgrade::half-validator-upgrade : 5372 TPS, 7162 ms latency, 11100 ms p99 latency,no expired txns
4. upgrading second batch to new version: c029334228a7e008509cf5fad9fd873020f26be8
compatibility::simple-validator-upgrade::rest-validator-upgrade : 7887 TPS, 4763 ms latency, 7400 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet ==> c029334228a7e008509cf5fad9fd873020f26be8 passed
Test Ok

Forge is running suite land_blocking on 45bd3dd2dcfd521d01718a2d4a87a0876401833a

Forge is running suite compat on testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a

:white_check_mark: Forge suite land_blocking success on 45bd3dd2dcfd521d01718a2d4a87a0876401833a

performance benchmark with full nodes : 7809 TPS, 5081 ms latency, 8400 ms p99 latency,no expired txns
Test Ok

Forge is running suite compat on testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a

Forge is running suite land_blocking on 45bd3dd2dcfd521d01718a2d4a87a0876401833a

:white_check_mark: Forge suite land_blocking success on 45bd3dd2dcfd521d01718a2d4a87a0876401833a

performance benchmark with full nodes : 7969 TPS, 4979 ms latency, 6900 ms p99 latency,no expired txns
Test Ok

Forge is running suite compat on testnet ==> 3e048f6b9a300f681b1f14f3ff515e9de209750d

Forge is running suite land_blocking on 3e048f6b9a300f681b1f14f3ff515e9de209750d

:white_check_mark: Forge suite land_blocking success on 3e048f6b9a300f681b1f14f3ff515e9de209750d

performance benchmark with full nodes : 8045 TPS, 4928 ms latency, 7800 ms p99 latency,no expired txns
Test Ok

github-actions[bot] avatar Sep 14 '22 18:09 github-actions[bot]

Forge is running suite land_blocking on cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge is running suite compat on testnet ==> cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

:x: Forge suite compat failure on testnet ==> cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge test runner terminated:
Trailing Log Lines:
             at /aptos/crates/aptos/src/main.rs:20:5
  33: core::ops::function::FnOnce::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4042-1662755346-testnet","timestamp":"2022-09-09T20:42:54.033191Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: Some(4), start_time: Some(Time(2022-09-09T20:32:59Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:260"},"thread_name":"main","hostname":"forge-compat-pr-4042-1662755346-testnet","timestamp":"2022-09-09T20:42:54.052974Z","message":"Deleting namespace forge-compat-pr-4042: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:368"},"thread_name":"main","hostname":"forge-compat-pr-4042-1662755346-testnet","timestamp":"2022-09-09T20:42:54.053008Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4042"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

:x: Forge suite land_blocking failure on cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge test runner terminated:
Trailing Log Lines:
             at /aptos/crates/aptos/src/main.rs:20:5
  33: core::ops::function::FnOnce::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-e2e-pr-4042-1662760839-cfd7c49a63bd8950f1e0aa5de4911c578a","timestamp":"2022-09-09T22:11:30.249247Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: Some(6), start_time: Some(Time(2022-09-09T22:01:34Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:260"},"thread_name":"main","hostname":"forge-e2e-pr-4042-1662760839-cfd7c49a63bd8950f1e0aa5de4911c578a","timestamp":"2022-09-09T22:11:30.269415Z","message":"Deleting namespace forge-e2e-pr-4042: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:368"},"thread_name":"main","hostname":"forge-e2e-pr-4042-1662760839-cfd7c49a63bd8950f1e0aa5de4911c578a","timestamp":"2022-09-09T22:11:30.269442Z","message":"aptos-node resources for Forge removed in namespace: forge-e2e-pr-4042"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

:white_check_mark: Forge suite compat success on testnet ==> 2483246aefbfb5e4fd5b9e6096a985f301547ffe

Compatibility test results for testnet ==> 2483246aefbfb5e4fd5b9e6096a985f301547ffe (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : 6942 TPS, 4060 ms latency, 6400 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 2483246aefbfb5e4fd5b9e6096a985f301547ffe
compatibility::simple-validator-upgrade::single-validator-upgrade : 5852 TPS, 4632 ms latency, 5900 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 2483246aefbfb5e4fd5b9e6096a985f301547ffe
compatibility::simple-validator-upgrade::half-validator-upgrade : 6036 TPS, 4327 ms latency, 5800 ms p99 latency,no expired txns
4. upgrading second batch to new version: 2483246aefbfb5e4fd5b9e6096a985f301547ffe
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6952 TPS, 3864 ms latency, 7500 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet ==> 2483246aefbfb5e4fd5b9e6096a985f301547ffe passed
Test Ok

:white_check_mark: Forge suite land_blocking success on ed773159f49f6c9dc05f632ac1df2017b851dea4

performance benchmark with full nodes : 7635 TPS, 3891 ms latency, 5700 ms p99 latency,no expired txns
Test Ok

:white_check_mark: Forge suite land_blocking success on c029334228a7e008509cf5fad9fd873020f26be8

performance benchmark with full nodes : 7827 TPS, 5074 ms latency, 9000 ms p99 latency,no expired txns
Test Ok

:white_check_mark: Forge suite compat success on testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a

Compatibility test results for testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : 7420 TPS, 4907 ms latency, 7600 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 45bd3dd2dcfd521d01718a2d4a87a0876401833a
compatibility::simple-validator-upgrade::single-validator-upgrade : 6356 TPS, 5758 ms latency, 7200 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 45bd3dd2dcfd521d01718a2d4a87a0876401833a
compatibility::simple-validator-upgrade::half-validator-upgrade : 5417 TPS, 6205 ms latency, 8200 ms p99 latency,no expired txns
4. upgrading second batch to new version: 45bd3dd2dcfd521d01718a2d4a87a0876401833a
compatibility::simple-validator-upgrade::rest-validator-upgrade : 7098 TPS, 5137 ms latency, 10000 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a passed
Test Ok

:white_check_mark: Forge suite compat success on testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a

Compatibility test results for testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : 8574 TPS, 4324 ms latency, 6900 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 45bd3dd2dcfd521d01718a2d4a87a0876401833a
compatibility::simple-validator-upgrade::single-validator-upgrade : 5379 TPS, 6569 ms latency, 8900 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 45bd3dd2dcfd521d01718a2d4a87a0876401833a
compatibility::simple-validator-upgrade::half-validator-upgrade : 6125 TPS, 5912 ms latency, 8100 ms p99 latency,no expired txns
4. upgrading second batch to new version: 45bd3dd2dcfd521d01718a2d4a87a0876401833a
compatibility::simple-validator-upgrade::rest-validator-upgrade : 7652 TPS, 4604 ms latency, 7700 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a passed
Test Ok

Forge is running suite land_blocking on 6a4f07de2b87c3c49a5ab61f8221f9ac6ba94178

Forge is running suite compat on testnet ==> 6a4f07de2b87c3c49a5ab61f8221f9ac6ba94178

:white_check_mark: Forge suite land_blocking success on 6a4f07de2b87c3c49a5ab61f8221f9ac6ba94178

performance benchmark with full nodes : 8116 TPS, 4881 ms latency, 8400 ms p99 latency,no expired txns
Test Ok

github-actions[bot] avatar Sep 14 '22 19:09 github-actions[bot]

Forge is running suite land_blocking on cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge is running suite compat on testnet ==> cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

:x: Forge suite compat failure on testnet ==> cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge test runner terminated:
Trailing Log Lines:
             at /aptos/crates/aptos/src/main.rs:20:5
  33: core::ops::function::FnOnce::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4042-1662755346-testnet","timestamp":"2022-09-09T20:42:54.033191Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: Some(4), start_time: Some(Time(2022-09-09T20:32:59Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:260"},"thread_name":"main","hostname":"forge-compat-pr-4042-1662755346-testnet","timestamp":"2022-09-09T20:42:54.052974Z","message":"Deleting namespace forge-compat-pr-4042: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:368"},"thread_name":"main","hostname":"forge-compat-pr-4042-1662755346-testnet","timestamp":"2022-09-09T20:42:54.053008Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4042"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

:x: Forge suite land_blocking failure on cfd7c49a63bd8950f1e0aa5de4911c578a10ba20

Forge test runner terminated:
Trailing Log Lines:
             at /aptos/crates/aptos/src/main.rs:20:5
  33: core::ops::function::FnOnce::call_once
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-e2e-pr-4042-1662760839-cfd7c49a63bd8950f1e0aa5de4911c578a","timestamp":"2022-09-09T22:11:30.249247Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: Some(6), start_time: Some(Time(2022-09-09T22:01:34Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:260"},"thread_name":"main","hostname":"forge-e2e-pr-4042-1662760839-cfd7c49a63bd8950f1e0aa5de4911c578a","timestamp":"2022-09-09T22:11:30.269415Z","message":"Deleting namespace forge-e2e-pr-4042: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:368"},"thread_name":"main","hostname":"forge-e2e-pr-4042-1662760839-cfd7c49a63bd8950f1e0aa5de4911c578a","timestamp":"2022-09-09T22:11:30.269442Z","message":"aptos-node resources for Forge removed in namespace: forge-e2e-pr-4042"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

:white_check_mark: Forge suite compat success on testnet ==> 2483246aefbfb5e4fd5b9e6096a985f301547ffe

Compatibility test results for testnet ==> 2483246aefbfb5e4fd5b9e6096a985f301547ffe (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : 6942 TPS, 4060 ms latency, 6400 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 2483246aefbfb5e4fd5b9e6096a985f301547ffe
compatibility::simple-validator-upgrade::single-validator-upgrade : 5852 TPS, 4632 ms latency, 5900 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 2483246aefbfb5e4fd5b9e6096a985f301547ffe
compatibility::simple-validator-upgrade::half-validator-upgrade : 6036 TPS, 4327 ms latency, 5800 ms p99 latency,no expired txns
4. upgrading second batch to new version: 2483246aefbfb5e4fd5b9e6096a985f301547ffe
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6952 TPS, 3864 ms latency, 7500 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet ==> 2483246aefbfb5e4fd5b9e6096a985f301547ffe passed
Test Ok

:white_check_mark: Forge suite land_blocking success on ed773159f49f6c9dc05f632ac1df2017b851dea4

performance benchmark with full nodes : 7635 TPS, 3891 ms latency, 5700 ms p99 latency,no expired txns
Test Ok

:white_check_mark: Forge suite land_blocking success on c029334228a7e008509cf5fad9fd873020f26be8

performance benchmark with full nodes : 7827 TPS, 5074 ms latency, 9000 ms p99 latency,no expired txns
Test Ok

:white_check_mark: Forge suite compat success on testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a

Compatibility test results for testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : 7420 TPS, 4907 ms latency, 7600 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 45bd3dd2dcfd521d01718a2d4a87a0876401833a
compatibility::simple-validator-upgrade::single-validator-upgrade : 6356 TPS, 5758 ms latency, 7200 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 45bd3dd2dcfd521d01718a2d4a87a0876401833a
compatibility::simple-validator-upgrade::half-validator-upgrade : 5417 TPS, 6205 ms latency, 8200 ms p99 latency,no expired txns
4. upgrading second batch to new version: 45bd3dd2dcfd521d01718a2d4a87a0876401833a
compatibility::simple-validator-upgrade::rest-validator-upgrade : 7098 TPS, 5137 ms latency, 10000 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a passed
Test Ok

:white_check_mark: Forge suite compat success on testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a

Compatibility test results for testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : 8574 TPS, 4324 ms latency, 6900 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 45bd3dd2dcfd521d01718a2d4a87a0876401833a
compatibility::simple-validator-upgrade::single-validator-upgrade : 5379 TPS, 6569 ms latency, 8900 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 45bd3dd2dcfd521d01718a2d4a87a0876401833a
compatibility::simple-validator-upgrade::half-validator-upgrade : 6125 TPS, 5912 ms latency, 8100 ms p99 latency,no expired txns
4. upgrading second batch to new version: 45bd3dd2dcfd521d01718a2d4a87a0876401833a
compatibility::simple-validator-upgrade::rest-validator-upgrade : 7652 TPS, 4604 ms latency, 7700 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet ==> 45bd3dd2dcfd521d01718a2d4a87a0876401833a passed
Test Ok

Forge is running suite land_blocking on 6a4f07de2b87c3c49a5ab61f8221f9ac6ba94178

Forge is running suite compat on testnet ==> 6a4f07de2b87c3c49a5ab61f8221f9ac6ba94178

:white_check_mark: Forge suite compat success on testnet ==> 6a4f07de2b87c3c49a5ab61f8221f9ac6ba94178

Compatibility test results for testnet ==> 6a4f07de2b87c3c49a5ab61f8221f9ac6ba94178 (PR)
1. Check liveness of validators at old version: testnet
compatibility::simple-validator-upgrade::liveness-check : 8370 TPS, 4335 ms latency, 7100 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 6a4f07de2b87c3c49a5ab61f8221f9ac6ba94178
compatibility::simple-validator-upgrade::single-validator-upgrade : 5833 TPS, 6615 ms latency, 8700 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 6a4f07de2b87c3c49a5ab61f8221f9ac6ba94178
compatibility::simple-validator-upgrade::half-validator-upgrade : 5481 TPS, 6564 ms latency, 8700 ms p99 latency,no expired txns
4. upgrading second batch to new version: 6a4f07de2b87c3c49a5ab61f8221f9ac6ba94178
compatibility::simple-validator-upgrade::rest-validator-upgrade : 8038 TPS, 4553 ms latency, 7600 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet ==> 6a4f07de2b87c3c49a5ab61f8221f9ac6ba94178 passed
Test Ok

github-actions[bot] avatar Sep 14 '22 19:09 github-actions[bot]