pytest icon indicating copy to clipboard operation
pytest copied to clipboard

iter_markers order is top-to-bottom at module node/scope

Open iwanb opened this issue 5 years ago • 6 comments

Not entirely sure if it would be considered a bug, but it surprised me. I expect iter_markers to return the markers in bottom-to-top order (outwards from the test), but for the pytestmark variable this is not the case, it returns them in the list order.

import pytest

pytestmark = [
    pytest.mark.order(4),
    pytest.mark.order(3),
]

@pytest.mark.order(2)
@pytest.mark.order(1)
def test_marker(request):
    lowest = 0
    for mark in request.node.iter_markers("order"):
        order = mark.args[0]
        assert order > lowest
        lowest = order
FAILED test_marker.py::test_marker - assert 3 > 4

I expect this will also apply for pytestmark as a class variable.

iwanb avatar Jul 31 '20 16:07 iwanb

Can you explain a bit more why you expect this?

bluetech avatar Aug 01 '20 09:08 bluetech

I see pytestmark = [x, y] as equivalent to x(y(module)), like the marker decorators at the test level, so it would seem more consistent to use the same ordering.

It's also to align with the "closest" marker terminology of the API, it seems that the bottom marker in pytestmark is closer to the test than the top one (at least visually). And I expect the last marker in the list to "win" so to speak when calling get_closest_marker.

iwanb avatar Aug 01 '20 10:08 iwanb

It's a structural bug missed since quit early

Marker applications append to the pytestmark list, they really should use prepend instead

RonnyPfannschmidt avatar Aug 01 '20 14:08 RonnyPfannschmidt

Agreed.

Is this OK for 6.1 or do we consider this is would be a serious breaking change and wait for 7.0?

nicoddemus avatar Aug 01 '20 16:08 nicoddemus

We can consider adding a config option to control the behaviour, before 7.0, we can't change the default for 6.x

RonnyPfannschmidt avatar Aug 01 '20 17:08 RonnyPfannschmidt

I would rather we just change it for 7.0 instead of adding an option because I don't think an option would be useful, probably just confusing. 👍

nicoddemus avatar Aug 02 '20 18:08 nicoddemus