Port output-check-record feature to the nrunner

[beraldoleal]

The idea of the features --output-check-record and --output-check are valid and we have today a few use cases that explore those functionalities.

The "problem" with this logic is that it is being used also to generate the test-results/stdout, stderr, and output files, depending on the --output-check-record value. I can understand that we are using this logic today because we are limited by the subprocess.Popen() ability to generate both "combined" and "split" (stdout and stderr) and also to avoid calling the tests twice.

We could split this logic into 3:

1. Sometimes the users would like to run the avocado only to generate the ".data/" folder with stdout, stderr, and combined results. For this situation we could have a boolean option named --record-output and the code can be executed twice to accomplish that. This run is something like a "dry-run" and there is no need to save information on job-results folder;

2. A normal run that MUST be executed only once that will generate the job-results information and will not care about the --record-output

3. A run that will check for the expected values stored inside .data/. This could be done with something like --check-output, also a boolean variable that will use the same logic that we have today:

3.1 If output.expected is there, then run the subprocess generating a "combined" version; 3.2 if stdout.expected or stderr.expected are there, then run the subprocess generating a "both/split" version;

@clebergnu any comments?

[clebergnu]

@beraldoleal I think you've captured it all. Nothing to add for now.

[beraldoleal]

@clebergnu I'm trying to focus on issues related to our effort to make nrun feature complete.

I have added this label "nrun2run" here, but I'm thinking better now, and I'm not sure if this is a dependency for the Next Runner, right?

I just would like to double-check with you if we can remove this label "nrun2run" here.

[beraldoleal]

Since the legacy runner will be removed lets focus on the new architecture and implement a new way to provide the same feature.

[clebergnu]

I have a BluePrint under development that will propose this feature under the new Avocado architecture (nrunner). Stay tuned!