joystream icon indicating copy to clipboard operation
joystream copied to clipboard

Extend integratin test suite

Open bedeho opened this issue 2 years ago • 0 comments

Background

Our current suite of integration tests for our entire network is quite limited, and we would benefit from having more regression tests as we are now going into mainnet mode, and have to be even more careful about making runtime and network changes.

Proposal

  1. Create integration testing markdown document which will live in tests/network-tests: This will be a canonical document, to be maintained going forward, describing the current status of our testing suite. It should describe what major families of tests are, what they do (high level), and what nodes/tools are involved.
  2. Map out our current tests and populate document, Lezek123 and mnaamani review.
  3. Make issues for new tests needed to cover rollout of carthage, as described #3792 , in particular checking that Sudo works, and no other extrinsics can be invoked.
  4. Suggest, in a new issue, extension of our current test suite to ensure we cover most valuable and critical features which are under tested or untested. In particular tests that cover content directory are important, as it has new NFT stuff, itnegration between content directory and storage pallet, and also future CRT features. Bounty module has a v2 version which is totally uncovered as well. No need to prioritize issues or add carthage label, this can be done as a group later.

Addedum: having multiple nodes (argus, collossus) could be a worth while scenario extension, if not as a first iteration, as it may reveal unique problems, as pointed out by @kdembler .

┆Issue is synchronized with this Asana task by Unito

bedeho avatar Jul 18 '22 11:07 bedeho