aptos-core icon indicating copy to clipboard operation
aptos-core copied to clipboard

[forge] custom profiles & add processor latency metrics

Open rustielin opened this issue 4 months ago • 3 comments

Description

Add indexer processor metrics to Forge. This does not include metrics from the indexer processors written with the processor SDK.

Also add support for providing a non-default forge deployer profile. By default tests use the forge profile, which is configured here: https://github.com/aptos-labs/internal-ops/blob/main/infra/cli/commands/forge/profiles.ts. For components outside of aptos-core like the indexer processors, it uses the build tagged main. However there are cases where we want to test experimental processors as well. In this case to keep the configuration easy to manage, we recommend creating a new test profile i.e. forge-indexer-sdk-test-metrics (some concise descriptive name), which specifies an exact build

After this PR lands, devs can use the FORGE_INDEXER_DEPLOYER_PROFILE var in adhoc forge to trigger their tests with new profiles, which can include experimental builds of indexer processors

In the future we might want to expose the entire deployer config to aptos-core too, and expose all test configuration in a config file rather than rust in main.rs of the forge-cli itself.

Bonus:

  • Start attempting to tune the indexer_test suite success thresholds

How Has This Been Tested?

CI + workflow dispatch once it lands

Also spin up using the create command with a different profile minimal:

cargo run -p aptos-forge-cli -- operator create --namespace forge-rustielin-test --num-validators 1 --enable-indexer --deployer-profile minimal     

Key Areas to Review

Type of Change

  • [x] New feature
  • [ ] Bug fix
  • [ ] Breaking change
  • [ ] Performance improvement
  • [ ] Refactoring
  • [ ] Dependency update
  • [ ] Documentation update
  • [ ] Tests

Which Components or Systems Does This Change Impact?

  • [ ] Validator Node
  • [ ] Full Node (API, Indexer, etc.)
  • [ ] Move/Aptos Virtual Machine
  • [ ] Aptos Framework
  • [ ] Aptos CLI/SDK
  • [x] Developer Infrastructure
  • [ ] Move Compiler
  • [ ] Other (specify)

Checklist

  • [x] I have read and followed the CONTRIBUTING doc
  • [x] I have performed a self-review of my own code
  • [x] I have commented my code, particularly in hard-to-understand areas
  • [x] I identified and added all stakeholders and component owners affected by this change as reviewers
  • [x] I tested both happy and unhappy path of the functionality
  • [x] I have made corresponding changes to the documentation

rustielin avatar Oct 10 '24 00:10 rustielin