flaky
flaky copied to clipboard
Flaky decorators on individual tests have no effect if `--force-flaky` argument is passed
From README.rst
:
Pass --max-runs=MAX_RUNS and/or --min-passes=MIN_PASSES to control the behavior of flaky if --force-flaky is specified. Flaky decorators on individual tests will override these defaults.
However, flaky decorators on individual tests do not override these defaults.
To repro the issue, consider the following test file test.py
:
import pytest
@pytest.mark.flaky(max_runs=10)
def test_foo():
assert False
Then run pytest test.py --force-flaky
and observe the test was run the default 2 times instead of the 10 specified by the decorator.
Problem appears to be here:
https://github.com/box/flaky/blob/d29ca76d5a72845428a0716c74f8fa0d40d98532/flaky/flaky_pytest_plugin.py#L252
This means an individual test's decorators won't be translated into flaky attributes, as the test's flaky attributes have already been set to the defaults (or values set in CLI args) here:
https://github.com/box/flaky/blob/d29ca76d5a72845428a0716c74f8fa0d40d98532/flaky/flaky_pytest_plugin.py#L77-L82