[feature request] Global subtest function

[Conchylicultor]

Currently, when using subtest, one has to propagate the subtest object in each of the submethods, adding boilerplate:

def test_xyz(subtests):
  assert_fn(x, y, subtests)  # Propagate subtest to all sub functions


def assert_fn(x, y, subtests):
  with subtests.test('a'):
    with subtests.test('x'):
      assert_child_fn(x, subtests)  # Additional function with subtests
    with subtests.test('y'):
      assert_child_fn(x, subtests)

It would be nice if there was instead a global subtest function which could be called directly. The above code could be rewritten as

@pytest.mark.usefixture('subtest')
def test_xyz():
  assert_fn(x, y)  # No more subtest argument


def assert_fn(x, y):
  with pytest.subtest('a'):
    with pytest.subtest('x'):
      assert_child_fn(x)
    with pytest.subtest('y'):
      assert_child_fn(x)

Of course, calling pytest.subtests would raise an error if executed in a test which does not use the subtest fixture. But this would make it easier to make modular tests function.

If pytest.subtest is not possible, import subtests directly would be great too.

[nicoddemus]

Hi @Conchylicultor,

Thanks for the suggestion!

Contrary to pytest.warns or pytest.raises, subtest support needs to capture exceptions, suppress them, and store that somewhere. Making it a global function would mean to store those captured exceptions in global variable, which I'm not a big fan of. Internally we also need to invoke hooks to generate reports, and we can't access those globally.

So while I agree it would be nice, I'm not sure it is possible to implement it without some ugly hacks.

I'm leaving this open anyway to review this in the future. 👍

[RonnyPfannschmidt]

@nicoddemus once we can provide contextvars it may be easier, but we should ensure its not quickly turned into a easy abused global

im currently leaning towards not enabling it

[Conchylicultor]

In case someone else is interested, I've implemented a small wrapper around subtests which add https://github.com/pytest-dev/pytest-subtests/issues/45 and https://github.com/pytest-dev/pytest-subtests/issues/44 (this issue):

from etils import epy

with_subtests = epy.testing.with_subtests  # Needed to register the fixture for this file


@pytest.mark.usefixtures('with_subtests')
def test_dtype():

    with epy.testing.subtests.test("a"):  # Do not need to propagate a subtests object in sub-functions
        with epy.testing.subtests.test("b"):  # Nested subtest is reported as `a/b`
            assert 1 == 2

Test reported as ___________ test_abc [a/b] ___________

As part of https://github.com/google/etils