Display of parametrized tests can be improved

[harmin-parra]

trafficstars

When I execute this test function:

@pytest.mark.parametrize("login, password, description", [("aaa", "111", "hello")])
def test_1(login, password, description):
    pass

The pytest-html report displays this test name: test/param_module_test.py::test_1[aaa-111-hello]

Wouldn't be better to display this? test/param_module_test.py::test_1(aaa, 111, hello)

[BeyondEvil]

That comes from pytest itself (part of nodeid), and I'm extremely reluctant to modify it.

[wayne-manselle-apex]

Also - if you're willing to take advantage of the hooks pytest-html has for customizing your report, this is well within the user's ability to clean up.

https://pytest-html.readthedocs.io/en/latest/user_guide.html#modifying-the-results-table

Here's an example of what can be done with a few standard tweaks and taking advantage of @pytest.mark.parametrize's ID parameter.

[Stefanhg]

It is a shame that pytest runner views a parametrized test as a seperate testcase but I would imagine it has to do with the way the runtest flow is.

@BeyondEvil In my case i want a parametrized test reproduced as a tree. I'm curious if you can confirm this is possible to do with hooks for pytest-html? Still trying to work out which hooks to hook onto and play with.

Here is a example of what how I am trying to do it:

TestsEqual/
├─ test_a_equal
├─ test_equal_1/
│  ├─ test_equal_1::a
│  ├─ test_equal_1::b
├─ test_b_equal
TestsNotEqual/
├─ test_a_not_equal
├─ test_b_not_equal

The test_equal_1::a and test_equal_1::b is parametrized testcase, TestsEqual, TestsNotEqual is a class and the rest is just testcases

[BeyondEvil]

I'm not entirely sure what you're trying to do w.r.t. the actual HTML report.

A mockup would help.

[Stefanhg]

I tried to recreate what i want in a image. It looks dumb i know but it is a simplified example

My goals:

• I want to group each testsuite and testcases(maybe also a second separation so each file has it's own section
• Paranetrized tests are each represented as a subtest of the main testcase as shown in picture

It could be done in a simple way like seperating it with a big fat black like, the only goal for me is to visually see a file, testcase and testsuite seprated and not combining testsuite, testcase and parametrized tests into a single "Test" field.

So this would give the "tree" flow as shown above in my other comment.

There are so many ways to do it:

• Seperate each file table so each file has it's own table
• Combine parametrized tests into a single testcase and then have a table under the testcase where it include the parametrized tests

The part where it has a table/area under the testcase is something i want to take more advantage of to include photos and such as way pytest-html work with linked photos or just including it in the table does not work in my case.

[BeyondEvil]

I wonder if this would be a first step to accomplish what you want: https://github.com/pytest-dev/pytest-html/pull/177

I think you can accomplish what you want, maybe not all of it but some of it, today. But it would require multiple hooks and a lot of code. Both python and HTML.

[Stefanhg]

You are right. Specially #33 is what i want except the parameterise I am just unsure how it #33 would be put together with #177 as here we split the items by giving them an value but you cannot give a module and class a value.

I think maybe the first step might actually to split some of the functions in the basereport.py so you could override them. I am still working on figuring out how everything is working but from what I have concluded then it would be a good idea for basereportBase.Report.pytest_runtest_logreport to call subfunctions instead of handling the cells, links, data and so on in the function. Then you can override and modify them much easier. Right now If i had to customize my html i have to make a whole replacement of the pytest_runtest_logreport.

Do you think it would be a good idea if i did that? If so I could create a task and start working on that.

[BeyondEvil]

You are right. Specially #33 is what i want except the parameterise I am just unsure how it #33 would be put together with #177 as here we split the items by giving them an value but you cannot give a module and class a value.

I think maybe the first step might actually to split some of the functions in the basereport.py so you could override them. I am still working on figuring out how everything is working but from what I have concluded then it would be a good idea for basereportBase.Report.pytest_runtest_logreport to call subfunctions instead of handling the cells, links, data and so on in the function. Then you can override and modify them much easier. Right now If i had to customize my html i have to make a whole replacement of the pytest_runtest_logreport.

Do you think it would be a good idea if i did that? If so I could create a task and start working on that.

I'm happy to review any PR that makes the code more extensible.

Keep in mind that for some of the things that happen in logreport and _process_report the order is very important.

I would also encourage you to take a TDD approach to the changes. :blush: