Help - How to preserve parallelism also if there are tests that needs to be executed alone?

[andreabisello]

Every test needs to be atomic. Tests should not affect other tests. This is clear. But there are conditions that are changed by test execution, and you cannot stop this. I'm talking about counters. For example, you have a totally indipendent test that increment a counter. You have another test that check that this counter is increased correctly. In order to test this counter increment, you make something that increment the counter and then you check the counter. The problem is when by the parallelism also all other test increment the same counter, and you cannot handle this.

i make an example.

Your software handle a coffee machine. You have a coffee cup counter. You want to test that after a coffee is done, the counter decreased by one. In order to make this test atomic, you

1. get the number of coffee cup
2. make a coffee
3. test the number of coffee cup is one less before

No problem with this.

The problem is when using xdist every other test use another coffee cup. Running in parallel, these test

1. make a normal coffee
2. make a long coffee
3. check coffee cup counter

can be execute IN THE SAME TIME, so "check coffee cup counter" can fail because while the test makes a coffee to check the counter is decresed by one, another test is making a coffee decreasing the counter by one making the "check cofree cup counter" fails.

so the request can be to "mark" "check coffee cup counter" as to be executed alone, when all the other tests are done.

there is something wrong in my test suite implementation? any suggestion?

thanks

[RonnyPfannschmidt]

Each worker is a indent process, if you need independent external resources, you need to provide acquisition /provisioning

[andreabisello]

@RonnyPfannschmidt Thanks for your anwer. What about acquisition/provisioning? can you explain to me better? thanks

[RonnyPfannschmidt]

acquisition /provisioning is a general name for processes used to obtain shared/exclusive access to resources

simplest example would be locks, but it can go up to local deployment of services/processes

[andreabisello]

thanks @RonnyPfannschmidt so a feature that mark test as to be not parallelized by xdist but to be executed sequentially can be a good feature for xdist?

[RonnyPfannschmidt]

it could, but nobody has taken a look at adding the proper side-channels for that

[kurt-cb]

actually such a feature does exist, but maybe it was not there before.

you can use the group scheduler and mark the interdependent tests with a group tag. They will be executed serially on a single node.

[andreabisello]

@kurt-cb thanks. I was also thinking that i some fixture regardings workers existed, i could inject that fixture into the test and put the tast in wait until "other" workers has nothing more to do. but for now i found only worker_id fixture inside plugin.py . If a test could know how much tests are in queue for the workers, then they can wait until workers has nothing to do, and add the test to a worker. in this way, some test could be marked as "sequential". What do you think?