feature: each test starts with a unique seed. `--randomly-use-same-seed-per-test` for old behavior

[brycedrennan]

Resolves #600

I'm tempted to make it the default behavior but this is safer.

[brycedrennan]

It's done but there is an inconsistency with how seeding works with faker. I'm not sure what would be least surprising for someone given the docs from faker. https://github.com/pytest-dev/pytest-randomly/blob/8b80740efd50c2785c8471cefba5fb7448e410f7/tests/test_pytest_randomly.py#L674-L692

https://faker.readthedocs.io/en/master/pytest-fixtures.html#seeding-configuration

[alexandrupatrascu]

hi folks, any chance this gets merged and released soon? reason why I'm asking is that I'm running pytest-rerunfailures alongside pytest-randomly and the re-runs use the same seed ID, causing the tests to re-use the same test data and not generate new inputs, hence they keep on failing.

thank you for the help!

[brycedrennan]

Sounds like you should change your test to either work with the same seed of data or change the data generator to not generate invalid data? I don't believe the changes proposed here would even help with your situation either way.

[tamird]

This would be very useful indeed.

In the meantime I worked around this in my own project using

@pytest.fixture(autouse=True)
# autouse fixtures with leading underscores are ignored. See https://github.com/pytest-dev/pytest/issues/12404.
def emulate_pytest_randomly_pull_request_617(request: pytest.FixtureRequest) -> None:  # noqa: PT004
    # pytest-randomly sets `randomly_seed` to an integer at pytest_configure time. See
    # https://github.com/pytest-dev/pytest-randomly/blob/8a3a241/src/pytest_randomly/__init__.py#L138.
    #
    # We include that integer to preserve the behavior of an explicit `--randomly-seed <int>` flag
    # passed on the command line while also giving each test a unique seed.
    request.config.option.randomly_seed += int(hashlib.sha512(request.node.nodeid.encode()).hexdigest(), base=16)

[adamchainz]

Thank you for this @brycedrennan . I have eventually got around to looking at it, and I decided it would be easiest to redo the PR, with you recorded as coauthor, in #687. I went for avoiding an option and updating the tests to expect certain fixed random values.

I think this is a great change, and will be combining it with #686 for a new major release! 🥳

[brycedrennan]

thanks! glad it made it in!