polkadot icon indicating copy to clipboard operation
polkadot copied to clipboard

Async backing integration & load test suite

Open sandreim opened this issue 2 years ago • 3 comments

The following need to be covered with Zombienet CI tests:

  • [x] Network protocol upgrade deploying both master and async branch (compatibility)
  • [x] Runtime upgrade while running both master and async backing branch nodes
  • [x] Async backing test with a mix of collators collating via async backing and sync backing.

More advanced load testing on Versi. So far we only tested with empty parachain blocks. We should explore some options to have the parachains fully fill their blocks with transactions or use undying test collator to burn CPU and create large PoVs.

The load tests need to produce a grid of performance numbers with async backing enabled. The following metrics need to be covered:

  • parachain block times
  • finality lag
  • TPS

The input parameters for this test will be:

  • number of parachains
  • number of paravalidators
  • PoV size (transactions)

Additional input parameters: configuration needs to match kusama as closely as possible for the following:

  • relay_vrf_modulo_samples
  • n_delay_tranches
  • max_validators_per_core
  • needed_approvals

We need to build some automation if possible so that we can run these tests for different combinations of input parameters.

sandreim avatar Feb 24 '23 10:02 sandreim