Support parsing parameterized tests

[OddBloke]

Continuing the conversation from #31:

Actually have been meaning to add support for parsing the parameterized tests as well, just haven't had a chance.

I put together a PR for neotest-dotnet that did something very similar to what's needed here https://github.com/Issafalcon/neotest-dotnet/pull/20. The core of the change is discussed here https://github.com/nvim-neotest/neotest/discussions/24#discussioncomment-3741387 where the parsing is customised to run queries after the initial parse, so the initial queries remain the same and then we traverse the tree to check if there are parameters.

If you want to have a go at it, happy to help but if not, I'll get around to it myself at some stage :smile:

Originally posted by @rcarriga in https://github.com/nvim-neotest/neotest-python/issues/31#issuecomment-1306910211

[OddBloke]

I'm not sure if we'll be able to parse parameterized tests effectively using Treesitter, for at least one reason: pytest fixtures can be parameterized. Consider this test file:

import pytest


@pytest.fixture(params=["a", "b"])
def fixture_param(request):
    return request.param


def test_only_fixture(fixture_param):
    assert fixture_param == "a"


@pytest.mark.parametrize("x", ["y", "z"])
def test_local_param(fixture_param, x):
    assert fixture_param == "a"
    assert x == "y"


@pytest.mark.parametrize("x", ["y", "z"])
@pytest.mark.parametrize("f", ["g", "h"])
def test_multiple_local_param(fixture_param, x, f):
    assert fixture_param == "a"
    assert x == "y"
    assert f == "g"

This produces the following tests (per --collect-only):

<Module test_params.py>
  <Function test_only_fixture[a]>
  <Function test_only_fixture[b]>
  <Function test_local_param[a-y]>
  <Function test_local_param[a-z]>
  <Function test_local_param[b-y]>
  <Function test_local_param[b-z]>
  <Function test_multiple_local_param[a-g-y]>
  <Function test_multiple_local_param[a-g-z]>
  <Function test_multiple_local_param[a-h-y]>
  <Function test_multiple_local_param[a-h-z]>
  <Function test_multiple_local_param[b-g-y]>
  <Function test_multiple_local_param[b-g-z]>
  <Function test_multiple_local_param[b-h-y]>
  <Function test_multiple_local_param[b-h-z]>

Fixtures are dynamically injected by pytest, so I think the only reliable way to know what parameters they will have is to invoke pytest. (Furthermore, fixtures don't have to be defined within the test file, so TS won't even have access to their code in many cases.)

[OddBloke]

And, actually, on further reflection, even without fixtures, it's not uncommon to do something like:

@pytest.mark.parametrize("path", pathlib.Path("./json/").glob("*.json"))
def test_json_files(path):
    assert json.load(open(path))["key"] = "value"

So the set of test instances would depend on the contents of json/.

[duncanam]

@OddBloke Is this a feature now? Currently VSCode does a really great job at looking at pytest parametrize cases and allowing you to split them out for debugging each individually which is essential, would be pretty slick if this allowed for neovim to enter that realm.