playwright-pytest
playwright-pytest copied to clipboard
Test collection order broken when using page fixture
Description
Given a folder contains multiple test modules each of which defining a test function that references the page fixture and one parametrized variable, when calling pytest --collect-only, then the list of test cases is not in module order but shuffled at seemingly random.
The issue occurs since pytest version 8.4.0. Tested with python 3.13.7, playwright 1.56.0 and pytest-playwright 0.7.1.
How to reproduce
pip install playwright pytest pytest-playwright
test_a.py
import pytest
@pytest.mark.parametrize("param", ["apple", "orange", "pear"])
def test_1(page, param):
assert True
test_b.py
import pytest
@pytest.mark.parametrize("param", ["dog", "cat", "mouse"])
def test_2(page, param):
assert True
Calling pytest --collect-only results in
<Package tests>
<Module test_a.py>
<Function test_1[chromium-apple]>
<Module test_b.py>
<Function test_2[chromium-dog]>
<Module test_a.py>
<Function test_1[chromium-orange]>
<Module test_b.py>
<Function test_2[chromium-cat]>
<Module test_a.py>
<Function test_1[chromium-pear]>
<Module test_b.py>
<Function test_2[chromium-mouse]>
Note: After removing the page fixture from one of the tests, the collection order is as expected:
<Package tests>
<Module test_a.py>
<Function test_1[apple]>
<Function test_1[orange]>
<Function test_1[pear]>
<Module test_b.py>
<Function test_2[chromium-dog]>
<Function test_2[chromium-cat]>
<Function test_2[chromium-mouse]>