[Bug] Plain pytest . does not work

[wenzeslaus]

Describe the bug

What I really like about pytest as opposed to to the grass.gunittest approach is that simple pytest . make it run. This is no longer the case as testsuite directories are included too and they require some additional setup. Simply using CI code does not work either, because that's set up for various edge cases and code coverage.

I would like to have the original option of running pytest in a simple way back. Is there a way to do that while working with the testsuite directories?

To reproduce

# Set up paths manually (skip with FHS)
export PYTHONPATH=$(./bin.x86_64-pc-linux-gnu/grass --config python-path)
export LD_LIBRARY_PATH=$(./bin.x86_64-pc-linux-gnu/grass --config path)/lib

# Run tests
pytest .

The above runs the tests, but since most them are testsuite, most files error and finally test_i_signatures stops the execution completely.

...
imagery/i.landsat.acca/testsuite/test_i_landsat_acca.py EEE                                                                                                                            imagery/i.landsat.toar/testsuite/test_i_landsat_toar.py EEEEEE                                                                                                                         imagery/i.maxlik/testsuite/test_i_maxlik.py EE                                                                                                                                         imagery/i.ortho.photo/i.ortho.elev/testsuite/test_i_ortho_elev.py EEE                                                                                                                  imagery/i.pca/testsuite/test_pca.py E                                                                                                                                                  imagery/i.rgb.his/testsuite/test_i_rgb_his.py EEEE                                                                                                                                     imagery/i.segment/testsuite/test_isegment.py E                                                                                                                                         imagery/i.signatures/testsuite/test_i_signatures.py 

Additional context

Having the testsuite tests executed by pytest is a good thing, but we don't want the brokenness of grass.gunittest transferred to pytest.

As I mentioned many times already, I'm the author of grass.gunittest, and I would do about every decision differently now.

[wenzeslaus]

@echoix What do you think? I can of course workaround this by executing the couple tests directories I'm interested in and relying on the CI for the rest, but sometimes running all tests is a good option to have.

[echoix]

I'll find the specific syntax later, but you can filter to the tests directories, or use a preset of options defined in a file that was made to simplify this on CI. I could also look up how to create an option/command/marker for having a list of tests run, like the ones that work directly without gunittest weirdness (caused by the way grass needs to be set up beforehand)

[wenzeslaus]

I tried again the CI command, but this time I installed all the pytest-related dependencies. The following worked.

          pytest \
            @.github/workflows/pytest_args_ci.txt \
            @.github/workflows/pytest_args_cov.txt \
            @.github/workflows/pytest_args_parallel.txt \
            -k 'not testsuite'

[wenzeslaus]

...but that's still little too complicated for showing someone how to test all after making a change. Or should we just focus on individual files locally and rely on CI for the rest? I'm thinking what to put, e.g., to the contributing guidelines.