temci icon indicating copy to clipboard operation
temci copied to clipboard

`temci report` should include (part of) error output, if the output format permits

Open sgraf812 opened this issue 2 years ago • 3 comments

Velcom's output format allows the specification of error measurements. Temci's report functionality should make use of that. Currently, I have to read both the exec as well as the report output to get error messages for failed runs.

sgraf812 avatar Nov 30 '21 10:11 sgraf812

Do you mean the velcom reporter?

parttimenerd avatar Nov 30 '21 11:11 parttimenerd

It appears that the reporter already has the required functionality implemented. And indeed if I dump the run_output.yaml, I can clearly see that the information is available under run.error.

But why don't I see any error messages in the JSON then?

`{"math/run": {}, "binarytrees/run": {}, "fannkuchredux/run": {}, "conways_game_of_life/run": {}, "BigTensorProduct/run": {}, ...`

It appears that the reporter only has the data from the StatsHelper. And that in turn is filled by run_processor.py, which does something strange: It takes result.error and mangles it somehow into some recorded_error that the StatsHelper can use, through its add_error method. I see self.stats_helper.add_error(id, result.recorded_error) here:

https://github.com/parttimenerd/temci/blob/a8d78cb52c248f1ae3f2469bbd0916b14ac9ea84/temci/run/run_processor.py#L328

But I don't think that result.recorded_error is ever set! So all we get are strange empty entries in the velcom output.

sgraf812 avatar Nov 30 '21 11:11 sgraf812

Do you the velcom reporter?

Not sure. Do I?

sgraf812 avatar Nov 30 '21 11:11 sgraf812