raven icon indicating copy to clipboard operation
raven copied to clipboard

[TASK] Supress Rook Output when a Test Sequence Yields No Results

Open dylanjm opened this issue 2 years ago • 0 comments


Issue Description

EDIT: Re-clarifying my issue as I discovered the feature I was asking for exists but doesn't work in the way I expected.

Is your feature request related to a problem? Please describe. The output for running some tests is sometimes confusing. When running

./run_tests --plugins --re multirun_sweep_opt

It will only run the HERON tests that match the regexp (as expected), but it also shows output for the rest of the plugins that are installed for raven. For example the above command will yield the following results:

Starting tests for plugin HERON ...
Running ROOK command: python /Users/mcdodj/Documents/projects/raven/rook/main.py --test-dir /Users/mcdodj/Documents/projects/HERON/tests --testers-dir /Users/mcdodj/Documents/projects/raven/framework/../scripts/TestHarness/testers,/Users/mcdodj/Documents/projects/HERON/tests/../src/Testers --add-non-default-run-types qsub --re multirun_sweep_opt ...
rook: found 18 test dirs under "/Users/mcdodj/Documents/projects/HERON/tests" ...
(1/2) Success( 25.39sec)/Users/mcdodj/Documents/projects/HERON/tests/integration_tests/mechanics/multirun_sweep_opt/MultiRun_Sweep
(2/2) Success(120.22sec)/Users/mcdodj/Documents/projects/HERON/tests/integration_tests/mechanics/multirun_sweep_opt/MultiRun_Opt

PASSED: 2
SKIPPED: 0
FAILED: 0
Tests passed for plugin HERON.

********************************************************************************

Starting tests for plugin TEAL ...
Running ROOK command: python /Users/mcdodj/Documents/projects/raven/rook/main.py --test-dir /Users/mcdodj/Documents/projects/TEAL/tests --testers-dir /Users/mcdodj/Documents/projects/raven/framework/../scripts/TestHarness/testers,/Users/mcdodj/Documents/projects/TEAL/tests/../src/Testers --add-non-default-run-types qsub --re multirun_sweep_opt ...
rook: found 1 test dirs under "/Users/mcdodj/Documents/projects/TEAL/tests" ...
invalid tester directory: /Users/mcdodj/Documents/projects/TEAL/tests/../src/Testers

PASSED: 0
SKIPPED: 0
FAILED: 0
Tests passed for plugin TEAL.

********************************************************************************

Starting tests for plugin ExamplePlugin ...
Running ROOK command: python /Users/mcdodj/Documents/projects/raven/rook/main.py --test-dir /Users/mcdodj/Documents/projects/raven/plugins/ExamplePlugin/tests --testers-dir /Users/mcdodj/Documents/projects/raven/framework/../scripts/TestHarness/testers,/Users/mcdodj/Documents/projects/raven/plugins/ExamplePlugin/tests/../src/Testers --add-non-default-run-types qsub --re multirun_sweep_opt ...
rook: found 1 test dirs under "/Users/mcdodj/Documents/projects/raven/plugins/ExamplePlugin/tests" ...

PASSED: 0
SKIPPED: 0
FAILED: 0
Tests passed for plugin ExamplePlugin.

********************************************************************************

This extra output seems unnecessary and might confuse some users who are expecting to only see output of the specific tests they asked for.

Describe the solution you'd like To only see the results of the tests I asked for.

Describe alternatives you've considered If needed, show the other plugins being skipped but reduce the verbosity of the output.


For Change Control Board: Issue Review

This review should occur before any development is performed as a response to this issue.

  • [ ] 1. Is it tagged with a type: defect or task?
  • [ ] 2. Is it tagged with a priority: critical, normal or minor?
  • [ ] 3. If it will impact requirements or requirements tests, is it tagged with requirements?
  • [ ] 4. If it is a defect, can it cause wrong results for users? If so an email needs to be sent to the users.
  • [ ] 5. Is a rationale provided? (Such as explaining why the improvement is needed or why current code is wrong.)

For Change Control Board: Issue Closure

This review should occur when the issue is imminently going to be closed.

  • [ ] 1. If the issue is a defect, is the defect fixed?
  • [ ] 2. If the issue is a defect, is the defect tested for in the regression test system? (If not explain why not.)
  • [ ] 3. If the issue can impact users, has an email to the users group been written (the email should specify if the defect impacts stable or master)?
  • [ ] 4. If the issue is a defect, does it impact the latest release branch? If yes, is there any issue tagged with release (create if needed)?
  • [ ] 5. If the issue is being closed without a pull request, has an explanation of why it is being closed been provided?

dylanjm avatar Oct 15 '21 21:10 dylanjm