pocket icon indicating copy to clipboard operation
pocket copied to clipboard

[Testing] Remove non-determinism from E2E Consensus Tests

Open Olshansk opened this issue 11 months ago • 1 comments

Objective

Avoid using timers or other non-deterministic approaches in E2E tests.

Origin Document

The following comment in #874 and as it relates to https://github.com/pokt-network/pocket/pull/948/files#diff-01dec4121ae8acb7a1f4bb72a6c2104827d2c2d2197eb5f45fa5c032ffba32cdR93:

Screenshot 2023-08-01 at 4 29 47 PM

Goals

  • Prevent non-determinism and flakiness from E2E tests
  • Create robust tooling to help iterate on E2E tests

Deliverable

  • [ ] Remove all the developer waits for ... milliseconds in all .feature files
  • [ ] Introduce alternate mechanisms mechanism to that waiting for a node to reach a certain height include and not limited to:
    • [ ] Polling for a specific height (with a max timeout)
    • [ ] Polling a health check endpoint (with a max timeout)

Non-goals / Non-deliverables

  • Introducing new tests
  • Introducing new functionality

General issue deliverables

  • [ ] Update any relevant local/global README(s)
  • [ ] Add or update any relevant or supporting mermaid diagrams

Testing Methodology

  • [ ] All tests: make test_all
  • [ ] E2E Tests: make test_e2e
  • [ ] LocalNet: verify a LocalNet is still functioning correctly by following the instructions at docs/development/README.md
  • [ ] k8s LocalNet: verify a k8s LocalNet is still functioning correctly by following the instructions here

Creator: @Olshansk Co-Owners: @0xBigBoss @dylanlott

Olshansk avatar Aug 02 '23 00:08 Olshansk