avocado icon indicating copy to clipboard operation
avocado copied to clipboard

nrunner: explicit report of test results that are missing.

Open clebergnu opened this issue 2 years ago • 0 comments

Is your feature request related to a problem? Please describe. The nrunner architecture allows for much more decoupled execution of tests and collection of its results. All existing spawners at this point, will instruct the tasks to send the task status messages to a server (-s $path_to_status_server).

But, a task may fail to send such information to a status server for a number of reasons (maybe network routing is broken). Or, future spawners may actively collect status messages from a different location (on non-networked task execution environments).

Describe the solution you'd like The job status report should clearly mark tests which failed to provide status information. One example would be to have a MISSING status:

 avocado run /bin/true /bin/uname 
JOB ID     : f6ce02825465da38dc6d2fca107c37ce74937697
JOB LOG    : /home/cleber/avocado/job-results/job-2022-04-25T12.45-f6ce028/job.log
 (1/2) /bin/true: STARTED
 (2/2) /bin/uname: STARTED
 (1/2) /bin/true: PASS (0.01 s)
 (2/2) /bin/uname: MISSING (120.00 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 | MISSING: 1
JOB HTML   : /home/cleber/avocado/job-results/job-2022-04-25T12.45-f6ce028/results.html
JOB TIME   : 1.33 s

Or extend the meaning of ERROR. ERROR is, so far, limited to the user's test error, and not an error on a layer above it.

Describe alternatives you've considered The only alternative I can think of is the current behavior simply "reconciles" the tests, marking those as "CANCEL"ed tests, but that is the essence of the problem.

clebergnu avatar Apr 25 '22 16:04 clebergnu