testify icon indicating copy to clipboard operation
testify copied to clipboard

suite: forward t as param to functions that declare it

Open Jerska opened this issue 2 years ago • 0 comments

Summary

Allow suite methods to accept t *testing.T as a parameter, like regular tests, to enable parallelism within a suite.

Changes

The change is simple: if the reflected method has one parameter, pass t as a param.

Motivation

The motivation is to partially fix #187 .

Right now, enabling any test of a suite as parallel using t.Parallel() makes Suite.T() unreliable in other tests of the suite. The reason for this is because Suite.T() is global to the suite, and having two tests running in parallel ends up with both tests modifying Suite.t at the same time.

By passing t to the test as argument, we follow the usual pattern of golang tests, and allow tests to be ran in parallel. Suite.T() & Suite.Assertions are still not usable reliably within those tests, but it's easy to use assert.Foo(t, ...) instead of s.Foo(...).

Ideally, this pattern should be applied to all the test-related lifecycle methods too (SetupTest / TeardownTest & BeforeTest / AfterTest). But I'd rather see if you're open to the change before going all in.

Example usage

Minimal example:

func (s *ParamSuite) TestSequential() {
	s.True(true)
}

func (s *ParamSuite) TestParallel_1(t *testing.T) {
	t.Parallel()
	time.Sleep(time.Second)
	assert.True(t, true)
}

func (s *ParamSuite) TestParallel_2(t *testing.T) {
	t.Parallel()
	time.Sleep(time.Second)
	assert.True(t, false) // No risk of this failure to be incorrectly reported as `TestParallel1` failing!
}

More relevant example:

type E2ETestSuite {
	suite.Suite
}

func (s *E2ETestSuite) SetupTest() {
	s.T().Parallel()
}

// All the e2e tests can now be run in parallel, they just all need to use the `t` param instead of `s.T()`

Related issues

#187

Jerska avatar Jan 12 '23 15:01 Jerska