pytest
pytest copied to clipboard
Arbitrary nested test functions
Description
I would like to have the ability to arbitrarily nest test functions in the same manner as facilitated by the pytest-describe plugin. While we can currently nest test functions using nested test classes, these nested classes impose limitations when accessing fields and properties from outer classes. My goal is to organize and group tests to mirror the structure of the code being tested. Specifically, I want a test class for each class (including nested and inherited classes) and a group (or multiple groups) of test functions for each function. Although pytest-describe provides this functionality, I believe it should be an integral part of pytest's general test discovery, organizational grouping, and structure.
What's the Problem This Feature Will Solve?
If implemented, this feature will allow:
- Organizing test functions in a manner that reflects the structure of the actual code under test.
- Shortening and enhancing the descriptiveness of test function names since outer functions up the hierarchy can provide additional context, eliminating the need to encode this context in the final test function. (See how it looks with the
pytest-describeplugin.) - Sharing data, initialization, and teardown procedures between tests at multiple hierarchy levels. Currently, this is only possible with class inheritance and, to some extent, with fixtures. However, fixtures cannot be used with test functions. For instance, I cannot share hypothesis strategies (passed to the
@givendecorator) between nested classes using fixtures.
Describe the Solution You'd Like
I would like the ability to arbitrarily nest test functions. pytest should be able to:
- Discover these nested test functions.
- Run groups of tests based on any function (anywhere in the hierarchy) that groups those test functions.
- When running a test function at the bottom of the hierarchy (or any other level), it should execute all the shared code up the hierarchy, meaning it should run all the necessary code up the hierarchy without running other groups of functions or other "sibling" test functions at the same level.
Alternative Solutions
I could use the pytest-describe plugin, but I believe this functionality should be part of the core pytest. I also found the pytest-relaxed plugin, which helps achieve similar functionality with inner classes by automatically passing all members of the outer class to the inner class. However, it seems to be incompatible with hypothesis.
A hard no for including this in core
It's using tracing to hack on collection and it is completely unclear how to integrate it with fixtures and parameterization
Python has no good Way to manage controlled namespaces beside classes