pytest-xdist icon indicating copy to clipboard operation
pytest-xdist copied to clipboard

Worker assign logic

Open SviatoslavKovalchuk opened this issue 1 year ago • 5 comments

Hey all! I already opened a similar ticket https://github.com/pytest-dev/pytest-xdist/issues/1031

Could some one please explain to me logic, whats going on under the hood when xdist assign (using @pytest.mark.xdist_group(name="example1")) group of test to worker ?

SviatoslavKovalchuk avatar Mar 18 '24 12:03 SviatoslavKovalchuk

Hey @SviatoslavKovalchuk,

I'm short on time, but I can point you to the source code: https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/scheduler/loadgroup.py

nicoddemus avatar Mar 18 '24 14:03 nicoddemus

Thank you for your quick response @nicoddemus I've already seen this a few times, and this is still not clear to me. I may have asked a little incorrect question. My main goal is to understand the logic of assigning test groups to workers, such as I describe in this ticket https://github.com/pytest-dev/pytest-xdist/issues/1031 and like the feature request here https://github.com/pytest-dev/pytest-xdist/issues/365

SviatoslavKovalchuk avatar Mar 18 '24 14:03 SviatoslavKovalchuk

The logic IIRC is simple: tests marked with the same xdist_group (and executing via loadgroup) will be sent to the same worker at runtime. You cannot control which worker that will be (so in the first run that might be gw1, the next run might be gw7, etc), but they will execute in the same worker.

nicoddemus avatar Mar 19 '24 15:03 nicoddemus

Can we control which worker will be assigned? And are you planning something like 'Lock tests to a specific worker'? And have a marker like @pytest.mark.xdist_group(name="example-group-one", worker="gw1") Thank you in advance @nicoddemus

SviatoslavKovalchuk avatar Mar 19 '24 15:03 SviatoslavKovalchuk

Can we control which worker will be assigned?

No, you cannot control directly.

And are you planning something like 'Lock tests to a specific worker'? And have a marker like @pytest.mark.xdist_group(name="example-group-one", worker="gw1")

None of the above is planned, however they should all be possible if you implement your own scheduler. You can use one of the builtin as basis: https://github.com/pytest-dev/pytest-xdist/tree/master/src/xdist/scheduler

nicoddemus avatar Mar 28 '24 22:03 nicoddemus