Parameterize class fixture with current index of another class parameter

[marshall7m]

trafficstars

Hi there,

Thank you for putting this package together. I hope that it will be able to handle the complexity of this use case since I haven't found a native pytest solution.

Background:

I'm writing integration tests that include a base pytest testing class (TestScenarioOne) that parametrizes a sub pytest testing class (TestIntegration) with the base pytest testing class data attribute. Within TestIntegration, a fixture (fixt_uses_data_len) is parametrized with a list (data_len_param) containing a range from 0 to the length of the current data parameter. Also within TestIntegration, there's a parameterized fixture called stage that needs to be grouped with every data parameter and every data_len_param.

Here's the directory tree:

.
├── test_integration.py
└── test_scenarios.py

Here's a redacted and shortened version of my scenario so your terminal isn't bombarded with hundreds of parametrized fixtures/tests while running pytest with --setup-plan :)

test_scenarios.py

from tests.bar import test_integration
from pytest_cases import param_fixtures, param_fixture

class TestScenarioOne(test_integration.TestIntegration):
    datasets = [
        {
            'point_1': 'bar'
        },
        {
            'point_2': 'foo',
            'point_3': 'baz'
        }
    ]

    params = []
    for d in datasets:
        params.append((d, list(range(0, len(d)))))
    data, data_len_param = param_fixtures("data, data_len_param", params, scope='class')

test_integration.py

from pytest_dependency import depends

class TestIntegration:

    @pytest.fixture(scope='class', params=['stage_1', 'stage_2'])
    def stage(self, request):
        return request.param

    @pytest.fixture(scope='class')
    def create_data(self, stage, data):
        yield 'data'

    @pytest.mark.dependency()
    def test_1_uses_data_param(self, request, stage, create_data):
        pass

    @pytest.mark.dependency()
    def test_2_uses_data_param(self, request, stage, create_data):
        depends(request, [f'{request.cls.__name__}::test_1_uses_data_param[{request.node.callspec.id}]'])
        pass

    @pytest.fixture(scope="class")
    def fixt_uses_data_len(self, create_data, data_len_param):
        yield 'data_len_param'

    @pytest.fixture(scope="class")
    def action(self, fixt_uses_data_len):
        yield 'action'
    
    @pytest.mark.dependency()
    def test_uses_data_len_param(self, request, stage, data, action, fixt_uses_data_len):
        depends(request, [f'{request.cls.__name__}::test_2_uses_data_param[{request.node.callspec.id}]'])
        pass

Here's the output when running pytest test_scenarios.py --setup-plan:

======================================== test session starts ========================================
platform linux -- Python 3.9.8, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /src
plugins: mock-3.6.1, dependency-0.5.1, cases-3.6.9, lazy-fixture-0.6.3
collected 6 items                                                                                   

test_scenarios.py 
      SETUP    C stage['stage_1']
      SETUP    C data
      SETUP    C create_data (fixtures used: data, stage)
        tests/bar/test_scenarios.py::TestScenarioOne::test_1_uses_data_param[stage_1] (fixtures used: create_data, data, request, stage)
        tests/bar/test_scenarios.py::TestScenarioOne::test_2_uses_data_param[stage_1] (fixtures used: create_data, data, request, stage)
      SETUP    C data_len_param
      SETUP    C fixt_uses_data_len (fixtures used: create_data, data_len_param)
      SETUP    C action (fixtures used: fixt_uses_data_len)
        tests/bar/test_scenarios.py::TestScenarioOne::test_uses_data_len_param[stage_1] (fixtures used: action, create_data, data, data_len_param, fixt_uses_data_len, request, stage)
      TEARDOWN C action
      TEARDOWN C fixt_uses_data_len
      TEARDOWN C create_data
      TEARDOWN C stage['stage_1']
      SETUP    C stage['stage_2']
      SETUP    C create_data (fixtures used: data, stage)
        tests/bar/test_scenarios.py::TestScenarioOne::test_1_uses_data_param[stage_2] (fixtures used: create_data, data, request, stage)
        tests/bar/test_scenarios.py::TestScenarioOne::test_2_uses_data_param[stage_2] (fixtures used: create_data, data, request, stage)
      SETUP    C fixt_uses_data_len (fixtures used: create_data, data_len_param)
      SETUP    C action (fixtures used: fixt_uses_data_len)
        tests/bar/test_scenarios.py::TestScenarioOne::test_uses_data_len_param[stage_2] (fixtures used: action, create_data, data, data_len_param, fixt_uses_data_len, request, stage)
      TEARDOWN C action
      TEARDOWN C fixt_uses_data_len
      TEARDOWN C data_len_param
      TEARDOWN C create_data
      TEARDOWN C data
      TEARDOWN C stage['stage_2']

======================================= no tests ran in 0.30s =======================================

Here's the expected output of pytest test_scenarios.py --setup-plan:

test_scenarios.py 
      SETUP    C stage['stage_1']
      SETUP    C data[{'point_1': 'bar'}]
      SETUP    C create_data (fixtures used: data, stage)
        tests/bar/test_scenarios.py::TestScenarioOne::test_1_uses_data_param[stage_1] (fixtures used: create_data, data, request, stage)
        tests/bar/test_scenarios.py::TestScenarioOne::test_2_uses_data_param[stage_1] (fixtures used: create_data, data, request, stage)
      SETUP    C data_len_param[0]
      SETUP    C fixt_uses_data_len (fixtures used: create_data, data_len_param)
      SETUP    C action (fixtures used: fixt_uses_data_len)
        tests/bar/test_scenarios.py::TestScenarioOne::test_uses_data_len_param[stage_1] (fixtures used: action, create_data, data, data_len_param, fixt_uses_data_len, request, stage)
      TEARDOWN C action
      TEARDOWN C fixt_uses_data_len
      TEARDOWN C create_data
      TEARDOWN C stage['stage_1']
      SETUP    C stage['stage_2']
      SETUP    C data[{'point_1': 'bar'}]
      SETUP    C create_data (fixtures used: data, stage)
        tests/bar/test_scenarios.py::TestScenarioOne::test_1_uses_data_param[stage_2] (fixtures used: create_data, data, request, stage)
        tests/bar/test_scenarios.py::TestScenarioOne::test_2_uses_data_param[stage_2] (fixtures used: create_data, data, request, stage)
      SETUP    C data_len_param[0]
      SETUP    C fixt_uses_data_len (fixtures used: create_data, data_len_param)
      SETUP    C action (fixtures used: fixt_uses_data_len)
        tests/bar/test_scenarios.py::TestScenarioOne::test_uses_data_len_param[stage_2] (fixtures used: action, create_data, data, data_len_param, fixt_uses_data_len, request, stage)
      TEARDOWN C action
      TEARDOWN C fixt_uses_data_len
      TEARDOWN C data_len_param
      TEARDOWN C create_data
      TEARDOWN C data
      TEARDOWN C stage['stage_2']

      SETUP    C stage['stage_1']
      SETUP    C data[{'point_2': 'foo','point_3': 'baz'}]
      SETUP    C create_data (fixtures used: data, stage)
        tests/bar/test_scenarios.py::TestScenarioOne::test_1_uses_data_param[stage_1] (fixtures used: create_data, data, request, stage)
        tests/bar/test_scenarios.py::TestScenarioOne::test_2_uses_data_param[stage_1] (fixtures used: create_data, data, request, stage)
      SETUP    C data_len_param[0]
      SETUP    C fixt_uses_data_len (fixtures used: create_data, data_len_param)
      SETUP    C action (fixtures used: fixt_uses_data_len)
        tests/bar/test_scenarios.py::TestScenarioOne::test_uses_data_len_param[stage_1] (fixtures used: action, create_data, data, data_len_param, fixt_uses_data_len, request, stage)
      TEARDOWN C action
      TEARDOWN C fixt_uses_data_len
      SETUP    C data_len_param[1]
      SETUP    C fixt_uses_data_len (fixtures used: create_data, data_len_param)
      SETUP    C action (fixtures used: fixt_uses_data_len)
        tests/bar/test_scenarios.py::TestScenarioOne::test_uses_data_len_param[stage_1] (fixtures used: action, create_data, data, data_len_param, fixt_uses_data_len, request, stage)
      TEARDOWN C action
      TEARDOWN C fixt_uses_data_len
      TEARDOWN C create_data
      TEARDOWN C stage['stage_1']
      SETUP    C stage['stage_2']
      SETUP    C data[{'point_2': 'foo','point_3': 'baz'}]
      SETUP    C create_data (fixtures used: data, stage)
        tests/bar/test_scenarios.py::TestScenarioOne::test_1_uses_data_param[stage_2] (fixtures used: create_data, data, request, stage)
        tests/bar/test_scenarios.py::TestScenarioOne::test_2_uses_data_param[stage_2] (fixtures used: create_data, data, request, stage)
      SETUP    C data_len_param[0]
      SETUP    C fixt_uses_data_len (fixtures used: create_data, data_len_param)
      SETUP    C action (fixtures used: fixt_uses_data_len)
        tests/bar/test_scenarios.py::TestScenarioOne::test_uses_data_len_param[stage_2] (fixtures used: action, create_data, data, data_len_param, fixt_uses_data_len, request, stage)
      TEARDOWN C action
      TEARDOWN C fixt_uses_data_len
      TEARDOWN C data_len_param[0]
      SETUP    C data_len_param[1]
      SETUP    C fixt_uses_data_len (fixtures used: create_data, data_len_param)
      SETUP    C action (fixtures used: fixt_uses_data_len)
        tests/bar/test_scenarios.py::TestScenarioOne::test_uses_data_len_param[stage_1] (fixtures used: action, create_data, data, data_len_param, fixt_uses_data_len, request, stage)
      TEARDOWN C action
      TEARDOWN C fixt_uses_data_len
      TEARDOWN C data_len_param[1]
      TEARDOWN C create_data
      TEARDOWN C data
      TEARDOWN C stage['stage_2']

As you can see my current implementation is way off from what is expected. I'm open to any suggestions even if it means refactoring the entire testing design structure.

Let me know if my post needs clarification or more context. Thanks!

[smarie]

Hi @marshall7m, thanks for the feedback ! I am deeply sorry, but I have trouble getting your example. Would you be kind enough to try rewriting it in the following form:

• desired test function signature
• for each parameter in the test function, how should it vary. Should it vary independently or at the same time as other parameters ?

For example

def test_foo(a, b, c):
    pass

Where

• a should take all values in a list
• b and c should vary FIRST as tuples in fixture A and THEN as a cross-product of fixture B and C.

Or similar. Indeed multi-parametrized test design is really far easier to read/understand bottom-up than top-down :)

[smarie]

Any progress on this @marshall7m ? Just to be sure you found a way out