avocado
avocado copied to clipboard
nrunner: explicit report of test results that are missing.
Is your feature request related to a problem? Please describe.
The nrunner architecture allows for much more decoupled execution of tests and collection of its results. All existing spawners at this point, will instruct the tasks to send the task status messages to a server (-s $path_to_status_server
).
But, a task may fail to send such information to a status server for a number of reasons (maybe network routing is broken). Or, future spawners may actively collect status messages from a different location (on non-networked task execution environments).
Describe the solution you'd like
The job status report should clearly mark tests which failed to provide status information. One example would be to have a MISSING
status:
avocado run /bin/true /bin/uname
JOB ID : f6ce02825465da38dc6d2fca107c37ce74937697
JOB LOG : /home/cleber/avocado/job-results/job-2022-04-25T12.45-f6ce028/job.log
(1/2) /bin/true: STARTED
(2/2) /bin/uname: STARTED
(1/2) /bin/true: PASS (0.01 s)
(2/2) /bin/uname: MISSING (120.00 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 | MISSING: 1
JOB HTML : /home/cleber/avocado/job-results/job-2022-04-25T12.45-f6ce028/results.html
JOB TIME : 1.33 s
Or extend the meaning of ERROR
. ERROR
is, so far, limited to the user's test error, and not an error on a layer above it.
Describe alternatives you've considered The only alternative I can think of is the current behavior simply "reconciles" the tests, marking those as "CANCEL"ed tests, but that is the essence of the problem.