pytest-xdist icon indicating copy to clipboard operation
pytest-xdist copied to clipboard

Enable configuring numprocesses's default `tx` command

Open puittenbroek opened this issue 1 year ago • 5 comments

Over the years I've used and introduced xdist whenever possibly to speed up pytest runs.

Usually just using the -n X notation was sufficient. But in our current application, we have to use the --tx notation to ensure we're using eventlet.

--tx '4*popen//execmodel=eventlet'

This is a lot to type if you 'just' want to speed up tests. And combining it with -n reverts it to just using popen. Ideally I'd configuring it to default to using popen//execmodel=eventlet and then up the processes using -n X notation.

So my feature request would be:

Enable configuring what 'executing method' -n actually uses. With popen being the default.

So that in your pytest.ini you can do something like this:

[pytest]
addopts = --default-tx popen//execmodel=eventlet

And then can add more worker as desired with the -n notation.

puittenbroek avatar Apr 12 '24 14:04 puittenbroek

Initially tx was primarily for remote hosts, there's no safe way to know the correct n

Would a setting for the default args help

RonnyPfannschmidt avatar Apr 12 '24 15:04 RonnyPfannschmidt

Initially tx was primarily for remote hosts, there's no safe way to know the correct n

Would a setting for the default args help

Depends a little bit what you mean with the default args :)

puittenbroek avatar Apr 16 '24 09:04 puittenbroek

default arg as in the tx used when using -n

RonnyPfannschmidt avatar Apr 16 '24 09:04 RonnyPfannschmidt

also heads up - there are considerations for removing gevent/eventlet support from execnet as their implementation in execnet is actively hostile to supporting asyncio as of now

RonnyPfannschmidt avatar Apr 16 '24 09:04 RonnyPfannschmidt

default arg as in the tx used when using -n

Yes that was exactly my intention! :)

also heads up - there are considerations for removing gevent/eventlet support from execnet as their implementation in execnet is actively hostile to supporting asyncio as of now

Hmm that is good to know. We're using nameko which uses eventlet under the mood. So using this tx helps us run everything in one session. But that would mean we'd need to split it up again :/ We'll see how to handle that once we get there :)

puittenbroek avatar Apr 16 '24 09:04 puittenbroek