aptos-core icon indicating copy to clipboard operation
aptos-core copied to clipboard

Published 20 hours ago • verified publisher icon aptos-labs

add test for STORAGE_WRITE_LIMIT_REACHED

Open msmouse opened this issue 3 years ago • 28 comments

Description

Test Plan

This change is Reviewable

msmouse avatar Oct 11 '22 23:10 msmouse

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 17eea1c64d3ea5d6ff621171f3c6d5945d448ff1

github-actions[bot] avatar Oct 16 '22 17:10 github-actions[bot]

Forge is running suite land_blocking on 17eea1c64d3ea5d6ff621171f3c6d5945d448ff1

github-actions[bot] avatar Oct 16 '22 17:10 github-actions[bot]

:white_check_mark: Forge suite land_blocking success on 17eea1c64d3ea5d6ff621171f3c6d5945d448ff1

performance benchmark with full nodes : 6836 TPS, 5822 ms latency, 7800 ms p99 latency,(!) expired 640 out of 2920020 txns
Test Ok

github-actions[bot] avatar Oct 16 '22 18:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> 17eea1c64d3ea5d6ff621171f3c6d5945d448ff1

Compatibility test results for 843b204dce971d98449b82624f4f684c7a18b991 ==> 17eea1c64d3ea5d6ff621171f3c6d5945d448ff1 (PR)
1. Check liveness of validators at old version: 843b204dce971d98449b82624f4f684c7a18b991
compatibility::simple-validator-upgrade::liveness-check : 7681 TPS, 5198 ms latency, 15200 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 17eea1c64d3ea5d6ff621171f3c6d5945d448ff1
compatibility::simple-validator-upgrade::single-validator-upgrade : 4770 TPS, 8948 ms latency, 14800 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 17eea1c64d3ea5d6ff621171f3c6d5945d448ff1
compatibility::simple-validator-upgrade::half-validator-upgrade : 4897 TPS, 8246 ms latency, 11700 ms p99 latency,no expired txns
4. upgrading second batch to new version: 17eea1c64d3ea5d6ff621171f3c6d5945d448ff1
Test Failed: Waiting for pod aptos-node-2-validator-0
Trailing Log Lines:
Compatibility test results for 843b204dce971d98449b82624f4f684c7a18b991 ==> 17eea1c64d3ea5d6ff621171f3c6d5945d448ff1 (PR)
1. Check liveness of validators at old version: 843b204dce971d98449b82624f4f684c7a18b991
compatibility::simple-validator-upgrade::liveness-check : 7681 TPS, 5198 ms latency, 15200 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 17eea1c64d3ea5d6ff621171f3c6d5945d448ff1
compatibility::simple-validator-upgrade::single-validator-upgrade : 4770 TPS, 8948 ms latency, 14800 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 17eea1c64d3ea5d6ff621171f3c6d5945d448ff1
compatibility::simple-validator-upgrade::half-validator-upgrade : 4897 TPS, 8246 ms latency, 11700 ms p99 latency,no expired txns
4. upgrading second batch to new version: 17eea1c64d3ea5d6ff621171f3c6d5945d448ff1
Test Failed: Waiting for pod aptos-node-2-validator-0


Swarm logs can be found here: See fgi output for more information.
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-compat-pr-4982-1665942887-843b204dce971d98449b82624f4f684","timestamp":"2022-10-16T18:11:11.967175Z","message":"Deleting namespace forge-compat-pr-4982: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-compat-pr-4982-1665942887-843b204dce971d98449b82624f4f684","timestamp":"2022-10-16T18:11:11.967195Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4982"}

failures:
    compatibility::simple-validator-upgrade

test result: FAILED. 0 passed; 1 failed; 0 filtered out

Failed to run tests:
Tests Failed
Error: Tests Failed
Debugging output:

github-actions[bot] avatar Oct 16 '22 18:10 github-actions[bot]

Forge is running suite land_blocking on 68dabb64e209a9e762805f90ee209b10cbcf3b73

github-actions[bot] avatar Oct 16 '22 23:10 github-actions[bot]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 68dabb64e209a9e762805f90ee209b10cbcf3b73

github-actions[bot] avatar Oct 16 '22 23:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> 68dabb64e209a9e762805f90ee209b10cbcf3b73

Forge test runner terminated:
Trailing Log Lines:
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge191-npzd7" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1665963513-843b204dce971d98449b82624f4f684","timestamp":"2022-10-16T23:49:09.255651Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-16T23:39:23Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:63"},"thread_name":"main","hostname":"forge-compat-pr-4982-1665963513-843b204dce971d98449b82624f4f684","timestamp":"2022-10-16T23:49:19.266683Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-16T23:39:23Z)), succeeded: None }"}
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge191-npzd7" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1665963513-843b204dce971d98449b82624f4f684","timestamp":"2022-10-16T23:49:19.334583Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-16T23:39:23Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-compat-pr-4982-1665963513-843b204dce971d98449b82624f4f684","timestamp":"2022-10-16T23:49:19.352496Z","message":"Deleting namespace forge-compat-pr-4982: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-compat-pr-4982-1665963513-843b204dce971d98449b82624f4f684","timestamp":"2022-10-16T23:49:19.352518Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4982"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

github-actions[bot] avatar Oct 16 '22 23:10 github-actions[bot]

:white_check_mark: Forge suite land_blocking success on 68dabb64e209a9e762805f90ee209b10cbcf3b73

performance benchmark with full nodes : 6855 TPS, 5794 ms latency, 8100 ms p99 latency,(!) expired 1020 out of 2928180 txns
Test Ok

github-actions[bot] avatar Oct 16 '22 23:10 github-actions[bot]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 68dabb64e209a9e762805f90ee209b10cbcf3b73

github-actions[bot] avatar Oct 17 '22 05:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> 68dabb64e209a9e762805f90ee209b10cbcf3b73

Forge test runner terminated:
Trailing Log Lines:
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge165-c6h55" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1665983570-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T05:22:44.759079Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T05:12:58Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:63"},"thread_name":"main","hostname":"forge-compat-pr-4982-1665983570-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T05:22:54.769109Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T05:12:58Z)), succeeded: None }"}
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge165-c6h55" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1665983570-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T05:22:54.841398Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T05:12:58Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-compat-pr-4982-1665983570-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T05:22:54.857430Z","message":"Deleting namespace forge-compat-pr-4982: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-compat-pr-4982-1665983570-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T05:22:54.857459Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4982"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

github-actions[bot] avatar Oct 17 '22 05:10 github-actions[bot]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 053accb1ef79d8a06616aa7feebe6a88558fc19c

github-actions[bot] avatar Oct 17 '22 16:10 github-actions[bot]

Forge is running suite land_blocking on 053accb1ef79d8a06616aa7feebe6a88558fc19c

github-actions[bot] avatar Oct 17 '22 16:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> 053accb1ef79d8a06616aa7feebe6a88558fc19c

Forge test runner terminated:
Trailing Log Lines:
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge242-4b4xj" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666024006-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T16:36:41.445724Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T16:26:55Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:63"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666024006-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T16:36:51.451490Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T16:26:55Z)), succeeded: None }"}
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge242-4b4xj" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666024006-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T16:36:51.529209Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T16:26:55Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666024006-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T16:36:51.550027Z","message":"Deleting namespace forge-compat-pr-4982: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666024006-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T16:36:51.550050Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4982"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

github-actions[bot] avatar Oct 17 '22 16:10 github-actions[bot]

:white_check_mark: Forge suite land_blocking success on 053accb1ef79d8a06616aa7feebe6a88558fc19c

performance benchmark with full nodes : 6830 TPS, 5825 ms latency, 8700 ms p99 latency,(!) expired 360 out of 2917060 txns
Test Ok

github-actions[bot] avatar Oct 17 '22 16:10 github-actions[bot]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 053accb1ef79d8a06616aa7feebe6a88558fc19c

github-actions[bot] avatar Oct 17 '22 18:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> 053accb1ef79d8a06616aa7feebe6a88558fc19c

Forge test runner terminated:
Trailing Log Lines:
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge239-24qc8" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666032420-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T18:57:44.346090Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T18:47:58Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:63"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666032420-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T18:57:54.355432Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T18:47:58Z)), succeeded: None }"}
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge239-24qc8" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666032420-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T18:57:54.436491Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T18:47:58Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666032420-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T18:57:54.451752Z","message":"Deleting namespace forge-compat-pr-4982: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666032420-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T18:57:54.451778Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4982"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:
NAME                                    READY   STATUS        RESTARTS   AGE
genesis-aptos-genesis-eforge239-24qc8   0/1     Terminating   0          10m

github-actions[bot] avatar Oct 17 '22 18:10 github-actions[bot]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 053accb1ef79d8a06616aa7feebe6a88558fc19c

github-actions[bot] avatar Oct 17 '22 21:10 github-actions[bot]

Forge is running suite land_blocking on 053accb1ef79d8a06616aa7feebe6a88558fc19c

github-actions[bot] avatar Oct 17 '22 21:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> 053accb1ef79d8a06616aa7feebe6a88558fc19c

Forge test runner terminated:
Trailing Log Lines:
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge85-tnxv7" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666042154-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T21:39:09.177596Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T21:29:23Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:63"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666042154-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T21:39:19.185839Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T21:29:23Z)), succeeded: None }"}
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge85-tnxv7" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666042154-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T21:39:19.256067Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T21:29:23Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666042154-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T21:39:19.275374Z","message":"Deleting namespace forge-compat-pr-4982: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666042154-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T21:39:19.275398Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4982"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:
NAME                                   READY   STATUS        RESTARTS   AGE
genesis-aptos-genesis-eforge85-tnxv7   0/1     Terminating   0          10m

github-actions[bot] avatar Oct 17 '22 21:10 github-actions[bot]

:white_check_mark: Forge suite land_blocking success on 053accb1ef79d8a06616aa7feebe6a88558fc19c

performance benchmark with full nodes : 6812 TPS, 5847 ms latency, 7800 ms p99 latency,(!) expired 280 out of 2909040 txns
Test Ok

github-actions[bot] avatar Oct 17 '22 21:10 github-actions[bot]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 31fb6edbf8deb4c1a387d09d26f1d816272bead7

github-actions[bot] avatar Oct 17 '22 23:10 github-actions[bot]

Forge is running suite land_blocking on 31fb6edbf8deb4c1a387d09d26f1d816272bead7

github-actions[bot] avatar Oct 17 '22 23:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> 31fb6edbf8deb4c1a387d09d26f1d816272bead7

Forge test runner terminated:
Trailing Log Lines:
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge115-lzk6z" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666049001-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T23:33:13.806955Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T23:23:28Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:63"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666049001-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T23:33:23.812748Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T23:23:28Z)), succeeded: None }"}
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge115-lzk6z" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666049001-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T23:33:23.883859Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-17T23:23:28Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666049001-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T23:33:23.900238Z","message":"Deleting namespace forge-compat-pr-4982: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666049001-843b204dce971d98449b82624f4f684","timestamp":"2022-10-17T23:33:23.900294Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4982"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:
NAME                                    READY   STATUS             RESTARTS   AGE
genesis-aptos-genesis-eforge115-lzk6z   0/1     ImagePullBackOff   0          9m59s

github-actions[bot] avatar Oct 17 '22 23:10 github-actions[bot]

:white_check_mark: Forge suite land_blocking success on 31fb6edbf8deb4c1a387d09d26f1d816272bead7

performance benchmark with full nodes : 6878 TPS, 5787 ms latency, 8100 ms p99 latency,(!) expired 240 out of 2937440 txns
Test Ok

github-actions[bot] avatar Oct 17 '22 23:10 github-actions[bot]

Forge is running suite land_blocking on 4960c36bffd432bc44ffe44055a882790fb6d90e

github-actions[bot] avatar Oct 18 '22 23:10 github-actions[bot]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 4960c36bffd432bc44ffe44055a882790fb6d90e

github-actions[bot] avatar Oct 18 '22 23:10 github-actions[bot]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> 4960c36bffd432bc44ffe44055a882790fb6d90e

Forge test runner terminated:
Trailing Log Lines:
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge75-6flv6" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666137144-843b204dce971d98449b82624f4f684","timestamp":"2022-10-19T00:02:38.582264Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-18T23:52:53Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:63"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666137144-843b204dce971d98449b82624f4f684","timestamp":"2022-10-19T00:02:48.590461Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-18T23:52:53Z)), succeeded: None }"}
Error from server (BadRequest): container "genesis" in pod "genesis-aptos-genesis-eforge75-6flv6" is waiting to start: trying and failing to pull image
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666137144-843b204dce971d98449b82624f4f684","timestamp":"2022-10-19T00:02:48.664796Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: None, start_time: Some(Time(2022-10-18T23:52:53Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666137144-843b204dce971d98449b82624f4f684","timestamp":"2022-10-19T00:02:48.681630Z","message":"Deleting namespace forge-compat-pr-4982: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-compat-pr-4982-1666137144-843b204dce971d98449b82624f4f684","timestamp":"2022-10-19T00:02:48.681654Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4982"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

github-actions[bot] avatar Oct 19 '22 00:10 github-actions[bot]

:white_check_mark: Forge suite land_blocking success on 4960c36bffd432bc44ffe44055a882790fb6d90e

performance benchmark with full nodes : 6947 TPS, 5691 ms latency, 9600 ms p99 latency,(!) expired 340 out of 2967120 txns
Test Ok

github-actions[bot] avatar Oct 19 '22 00:10 github-actions[bot]