[Forge] Add basic support for large db test in forge.

[grao1991]

Description

Add the capability to claim an existing PV using a specified tag.

Test Plan

Manually verified that this can successfully mount an existing EBS volume.

---

This change is 

[github-actions[bot]]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 10a99893ca9b7026d59807ee53cf9dab8d8430e1

• Grafana dashboard (auto-refresh)
• Humio Logs
• (Deprecated) OpenSearch Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

Forge is running suite land_blocking on 10a99893ca9b7026d59807ee53cf9dab8d8430e1

• Grafana dashboard (auto-refresh)
• Humio Logs
• (Deprecated) OpenSearch Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

Forge is running suite land_blocking on 4000aae42eafd7cd56f0cef849c38b180b988066

• Grafana dashboard (auto-refresh)
• Humio Logs
• (Deprecated) OpenSearch Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 4000aae42eafd7cd56f0cef849c38b180b988066

• Grafana dashboard (auto-refresh)
• Humio Logs
• (Deprecated) OpenSearch Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:white_check_mark: Forge suite compat success on 843b204dce971d98449b82624f4f684c7a18b991 ==> 4000aae42eafd7cd56f0cef849c38b180b988066

Compatibility test results for 843b204dce971d98449b82624f4f684c7a18b991 ==> 4000aae42eafd7cd56f0cef849c38b180b988066 (PR)
1. Check liveness of validators at old version: 843b204dce971d98449b82624f4f684c7a18b991
compatibility::simple-validator-upgrade::liveness-check : 7983 TPS, 4674 ms latency, 6600 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 4000aae42eafd7cd56f0cef849c38b180b988066
compatibility::simple-validator-upgrade::single-validator-upgrade : 5372 TPS, 7104 ms latency, 10200 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 4000aae42eafd7cd56f0cef849c38b180b988066
compatibility::simple-validator-upgrade::half-validator-upgrade : 4165 TPS, 8758 ms latency, 12600 ms p99 latency,no expired txns
4. upgrading second batch to new version: 4000aae42eafd7cd56f0cef849c38b180b988066
compatibility::simple-validator-upgrade::rest-validator-upgrade : 7669 TPS, 4869 ms latency, 8300 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for 843b204dce971d98449b82624f4f684c7a18b991 ==> 4000aae42eafd7cd56f0cef849c38b180b988066 passed
Test Ok

• Grafana dashboard
• Humio Logs
• (Deprecated) OpenSearch Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:white_check_mark: Forge suite land_blocking success on 4000aae42eafd7cd56f0cef849c38b180b988066

performance benchmark with full nodes : 7750 TPS, 5124 ms latency, 7900 ms p99 latency,no expired txns
Test Ok

• Grafana dashboard
• Humio Logs
• (Deprecated) OpenSearch Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 0350c24ae73a5c793ba4049db0882de13e268ecb

• Grafana dashboard (auto-refresh)
• Humio Logs
• (Deprecated) OpenSearch Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

Forge is running suite land_blocking on 0350c24ae73a5c793ba4049db0882de13e268ecb

• Grafana dashboard (auto-refresh)
• Humio Logs
• (Deprecated) OpenSearch Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:white_check_mark: Forge suite land_blocking success on 0350c24ae73a5c793ba4049db0882de13e268ecb

performance benchmark with full nodes : 7773 TPS, 5098 ms latency, 6900 ms p99 latency,no expired txns
Test Ok

• Grafana dashboard
• Humio Logs
• (Deprecated) OpenSearch Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:white_check_mark: Forge suite compat success on 843b204dce971d98449b82624f4f684c7a18b991 ==> 0350c24ae73a5c793ba4049db0882de13e268ecb

Compatibility test results for 843b204dce971d98449b82624f4f684c7a18b991 ==> 0350c24ae73a5c793ba4049db0882de13e268ecb (PR)
1. Check liveness of validators at old version: 843b204dce971d98449b82624f4f684c7a18b991
compatibility::simple-validator-upgrade::liveness-check : 8061 TPS, 4494 ms latency, 6000 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 0350c24ae73a5c793ba4049db0882de13e268ecb
compatibility::simple-validator-upgrade::single-validator-upgrade : 6075 TPS, 6235 ms latency, 8200 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 0350c24ae73a5c793ba4049db0882de13e268ecb
compatibility::simple-validator-upgrade::half-validator-upgrade : 5300 TPS, 7178 ms latency, 9400 ms p99 latency,no expired txns
4. upgrading second batch to new version: 0350c24ae73a5c793ba4049db0882de13e268ecb
compatibility::simple-validator-upgrade::rest-validator-upgrade : 8296 TPS, 4296 ms latency, 6500 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for 843b204dce971d98449b82624f4f684c7a18b991 ==> 0350c24ae73a5c793ba4049db0882de13e268ecb passed
Test Ok

• Grafana dashboard
• Humio Logs
• (Deprecated) OpenSearch Logs
• Test runner output
• Test run is land-blocking

[sitalkedia]

@grao1991 - Do you have a successful run for this test? If so, can you point to the link? Also, can you update the test plan accordingly?

[sitalkedia]

You might also want to add github workflow yaml file to let this run in continous mode?

[github-actions[bot]]

Forge is running suite land_blocking on 77dce1735eb65e12344d6ec9223639287f656e6a

• Grafana dashboard (auto-refresh)
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 77dce1735eb65e12344d6ec9223639287f656e6a

• Grafana dashboard (auto-refresh)
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> 77dce1735eb65e12344d6ec9223639287f656e6a

Forge test runner terminated:
Trailing Log Lines:
USAGE:
    aptos genesis set-validator-configuration --username <USERNAME> --validator-host <VALIDATOR_HOST> --owner-public-identity-file <OWNER_PUBLIC_IDENTITY_FILE> --local-repository-dir <LOCAL_REPOSITORY_DIR> --full-node-host <FULL_NODE_HOST> --stake-amount <STAKE_AMOUNT>

For more information try --help
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4348-1664336432-843b204dce971d98449b82624f4f684","timestamp":"2022-09-28T03:51:15.714395Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: Some(6), start_time: Some(Time(2022-09-28T03:41:18Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-compat-pr-4348-1664336432-843b204dce971d98449b82624f4f684","timestamp":"2022-09-28T03:51:15.739104Z","message":"Deleting namespace forge-compat-pr-4348: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-compat-pr-4348-1664336432-843b204dce971d98449b82624f4f684","timestamp":"2022-09-28T03:51:15.739129Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4348"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:white_check_mark: Forge suite land_blocking success on 77dce1735eb65e12344d6ec9223639287f656e6a

performance benchmark with full nodes : 7352 TPS, 5324 ms latency, 12000 ms p99 latency,(!) expired 4160 out of 3143840 txns
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

Forge is running suite land_blocking on 3cbc5e67073fd9bb801882ee30c75de5090717a8

• Grafana dashboard (auto-refresh)
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> 3cbc5e67073fd9bb801882ee30c75de5090717a8

• Grafana dashboard (auto-refresh)
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:x: Forge suite compat failure on 843b204dce971d98449b82624f4f684c7a18b991 ==> 3cbc5e67073fd9bb801882ee30c75de5090717a8

Forge test runner terminated:
Trailing Log Lines:
USAGE:
    aptos genesis set-validator-configuration --username <USERNAME> --validator-host <VALIDATOR_HOST> --owner-public-identity-file <OWNER_PUBLIC_IDENTITY_FILE> --local-repository-dir <LOCAL_REPOSITORY_DIR> --full-node-host <FULL_NODE_HOST> --stake-amount <STAKE_AMOUNT>

For more information try --help
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:81"},"thread_name":"main","hostname":"forge-compat-pr-4348-1664404425-843b204dce971d98449b82624f4f684","timestamp":"2022-09-28T22:44:36.545349Z","message":"Genesis status: JobStatus { active: Some(1), completion_time: None, conditions: None, failed: Some(6), start_time: Some(Time(2022-09-28T22:34:38Z)), succeeded: None }"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:280"},"thread_name":"main","hostname":"forge-compat-pr-4348-1664404425-843b204dce971d98449b82624f4f684","timestamp":"2022-09-28T22:44:36.566233Z","message":"Deleting namespace forge-compat-pr-4348: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:388"},"thread_name":"main","hostname":"forge-compat-pr-4348-1664404425-843b204dce971d98449b82624f4f684","timestamp":"2022-09-28T22:44:36.566281Z","message":"aptos-node resources for Forge removed in namespace: forge-compat-pr-4348"}
Failed to run tests:
Genesis did not succeed
Error: Genesis did not succeed
Debugging output:

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:white_check_mark: Forge suite land_blocking success on 3cbc5e67073fd9bb801882ee30c75de5090717a8

performance benchmark with full nodes : 7599 TPS, 5213 ms latency, 12600 ms p99 latency,(!) expired 1180 out of 3246240 txns
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[grao1991]

@grao1991 - Do you have a successful run for this test? If so, can you point to the link? Also, can you update the test plan accordingly?

Not yet, some additional work are needed to make it work end to end.

[github-actions[bot]]

Forge is running suite land_blocking on f6f6670e857438ddb4c6c576d2c843aea19ab3d7

• Grafana dashboard (auto-refresh)
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

Forge is running suite compat on 843b204dce971d98449b82624f4f684c7a18b991 ==> f6f6670e857438ddb4c6c576d2c843aea19ab3d7

• Grafana dashboard (auto-refresh)
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:white_check_mark: Forge suite land_blocking success on f6f6670e857438ddb4c6c576d2c843aea19ab3d7

performance benchmark with full nodes : 7447 TPS, 5333 ms latency, 7800 ms p99 latency,(!) expired 80 out of 3179960 txns
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions[bot]]

:white_check_mark: Forge suite compat success on 843b204dce971d98449b82624f4f684c7a18b991 ==> f6f6670e857438ddb4c6c576d2c843aea19ab3d7

Compatibility test results for 843b204dce971d98449b82624f4f684c7a18b991 ==> f6f6670e857438ddb4c6c576d2c843aea19ab3d7 (PR)
1. Check liveness of validators at old version: 843b204dce971d98449b82624f4f684c7a18b991
compatibility::simple-validator-upgrade::liveness-check : 7398 TPS, 4945 ms latency, 7500 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: f6f6670e857438ddb4c6c576d2c843aea19ab3d7
compatibility::simple-validator-upgrade::single-validator-upgrade : 5579 TPS, 6741 ms latency, 8900 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: f6f6670e857438ddb4c6c576d2c843aea19ab3d7
compatibility::simple-validator-upgrade::half-validator-upgrade : 4944 TPS, 8427 ms latency, 14000 ms p99 latency,no expired txns
4. upgrading second batch to new version: f6f6670e857438ddb4c6c576d2c843aea19ab3d7
compatibility::simple-validator-upgrade::rest-validator-upgrade : 7677 TPS, 4794 ms latency, 8600 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for 843b204dce971d98449b82624f4f684c7a18b991 ==> f6f6670e857438ddb4c6c576d2c843aea19ab3d7 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[davidiw]

closing out stale PR