cscore icon indicating copy to clipboard operation
cscore copied to clipboard

Make sure the xUnit tests all run in parallel without problems

Open cs-util opened this issue 4 years ago • 1 comments

Currently when running all xUnit tests at once not all tests run through successfully

Some tests fail since they depend on internal singleton constructs, testing generic interfaces that internally use injection and because of this are affected by the other running tests. If the interface of the components is not modified the only way to manage to run all tests at once successfully is to separate the tests into fully independent processes without any shared memory, I guess/hope this is possible in xUnit but did not invest time into figuring out how to tell xUnit to run all tests fully separate in memory

Another type of test are the performance tests which fail because they expect the CPU to not be busy with running hundreds of other tests in parallel, maybe there is something in xUnit that allows specifying that a test should only be executed if no other tests are running at the same time? So queuing all performance tests to execute them linearly at the very end of a test run?

One bonus if this is fixed and all tests run reliably through each time they are executed at once is that the Stryker mutation test system can be used again

cs-util avatar Nov 25 '20 08:11 cs-util

should be a little easier to finish now, most of the tests run now successfully when executed together

The maim problem still are the ones that use external dependencies like APIs, use internal singleton systems or persist data to a non test specific location

cs-util avatar Dec 04 '20 08:12 cs-util

Now moved all tests that have external dependencies or state that is shared between multiple tests into an integrationTest namespace and excluded them from being used by Stryker so that Stryker only uses the real unit tests that always pass

cs-util avatar Sep 28 '23 07:09 cs-util