consensus-specs icon indicating copy to clipboard operation
consensus-specs copied to clipboard

Extend broad-spectrum block tests

Open ralexstokes opened this issue 4 years ago • 0 comments

Ideas to extend the test type added in #2560

  • Attester slashings for both types of slashing conditions (right now it only duplicates the target)
  • Ensure all possible validator states are covered (refer: https://github.com/ethereum/consensus-specs/pull/2560#discussion_r695249781)
  • Ensure coverage of random operations (e.g. random.randrange(1, spec.MAX_OP) instead of random.randrange(1, spec.MAX_OP + 1))
  • Scenario: test regular (randomized) block transition to leaking
  • Scenario: test regular (randomzied) block transition to leaking and back to recovery
  • The (current, next) sync committees are the same in all Altair tests -- have a few tests that rotate into new committees
  • Adversarial (or almost) eth1-data voting

Ideas to extend the testing infra itself

  • Use stronger typing for underlying test gen/harness (right now, just plain python data with weak typing) refer: https://github.com/ethereum/consensus-specs/pull/2560#discussion_r695967907
  • Mainnet altair tests take a notable amount of time. Investigate and optimize where feasible.
  • More flexible deposit testing -- right now, the format of the test doesn't allow for an eth1 data voting process to take place, which limits the type of deposit testing we can fit into the skeleton of (pre_state, [blocks], post_state) format

ralexstokes avatar Aug 25 '21 17:08 ralexstokes