Testing Architecture

[MSevey]

Overview

This issue aims to document some testing structure clean up that should help improve the testing of the rollkit repo.

Goals

1. Minimal to zero hard coded sleeps
2. Clear documentation of testing helpers
3. Testing of the test helps
4. Minimal to zero use of waitgroups or channels in testing that are not confined to the scope of a single function or test

Stage 1 - Refactoring

As a first step, we should refactor some or all of the testing helpers to the top of the file or into common files so that they are not lost in an ever growing test file.

For example, all the Node related helpers createNodes createNode startNodes etc should be moved to the top of the file or pulled out of the full_integration_test.go and put in a testing_helpers_test.go or something.

Stage 2 - Sleeps and Waitgroups

Next we should work to remove sleeps and waitgroups from the helpers and instead make use of retries.

Part of this will be the need to understand which helpers, and what scope of the helpers is expected to be blocking. Or to put another way, we should be able to very clearly define the expected state after a helper is called. This in turn means we should be able to write tests or our helpers to ensure that they are doing this.

Writing tests of our helpers, allows developers to confidently call the helpers and not have to worry about adding addition sleeps or waits in their tests.

Stage 3 - Test clean up

Now that we have nice clean, well scoped helpers, we can revisit sleeps and waits in our tests themselves and determine how to update these tests to not require sleeps and waits.

[tuxcanfly]

Flagging TestValidatorSetHandlingBased in full_client_test.go as a example of test using Waitgroup, there are others but ran into a timeout due to -test.timeout=30s, would be great to remove the waitgroup and speed this test up, among others.

[Manav-Aggarwal]

Related: https://github.com/rollkit/rollkit/issues/1321

[MSevey]

Next Steps on this issue should be to do a code audit since it has been a year and there has been a lot of updates.

Result of the audit should be recommendations on architecture updates and status of issues referenced above.