Exception in scenario.py when executing tests in parallel threads

[KrystianOcado]

trafficstars

Hi!

I would like to report an issue in pytest-bdd:

pytest-bdd 3.2.1 pytest 5.4.1 Python 3.7.5

When executing tests in parallel threads, for example using using threads pytest-parallel, a race condition can occur in file pytest_bdd/scenario.py function find_argumented_step_fixture_name:

name = 'foobar'
type_ = 'given'
fixturemanager = <_pytest.fixtures.FixtureManager object at 0x7f8d6f0a4bd0>
request = <FixtureRequest for <Function foobar>>
    def find_argumented_step_fixture_name(name, type_, fixturemanager, request=None):
        """Find argumented step fixture name."""
        # happens to be that _arg2fixturedefs is changed during the iteration so we use a copy
>       for fixturename, fixturedefs in list(fixturemanager._arg2fixturedefs.items()):
E       RuntimeError: dictionary changed size during iteration
/lib/python3.7/site-packages/pytest_bdd/scenario.py:51: RuntimeError

It seems that what happens is, while a copy of dictionary is created here: list(fixturemanager._arg2fixturedefs.items()) fixture dictionary changes.

This can only be reproduced using Python 3, because dict.items() returns an iterator in Python 3, not a list of keys like in Python 2.

This issue should not appear in pytest-xdist because it uses multiprocessing.

I looked around and this code could use dict.copy().items(), instead of list(dict.items()), which should be thread-safe.

I understand that pytest-parallel might not be officially supported in pytest-bdd, but it would still be good to fix this issue.

I can fix it, but reproducing it consistently in test environment is really hard, so it would be hard to prove my fix.

[youtux]

Possibly related to https://bugs.python.org/issue40327. @KrystianOcado, can you provide an mcve to reproduce the issue? It's fine if it doesn't always show, as long as one in many runs of the example shows it.

[jruizaranguren]

I have run some tests with:

pytest-bdd 4.02 pytest-parallel 0.1.0

Tests are executed nicely, but then it hangs.

[Enigmaderockz]

KrystianOcado - can you please tell me how did you implement pytest-parallel to execute pytest-bdd tests in parallel?

[KrystianOcado]

Hi @youtux !

Sorry I missed your message. I'm unable to provide mvce, even tough I tried. I was not able to reproduce it a fresh project. It looks like I was the only one to experience it, so I will close this issue. Sorry again and thanks!

HI @Enigmaderockz !

Dependencies:

pytest-bdd = "==6.0.0"
pytest-parallel = "==0.1.1"

pytest.ini

addopts =
    --workers=1
    --tests-per-worker=20

[Enigmaderockz]

thanks @KrystianOcado . can you please tell me what command you are using to run pytest-bdd test cases parallely using pytest-parallel? I am using pytest-bdd -sv test_filename.py.

[KrystianOcado]

Hi @Enigmaderockz !

I use pytest command to run tests.