temci
temci copied to clipboard
`temci report` should include (part of) error output, if the output format permits
Velcom's output format allows the specification of error
measurements. Temci's report functionality should make use of that. Currently, I have to read both the exec
as well as the report
output to get error messages for failed runs.
Do you mean the velcom reporter?
It appears that the reporter already has the required functionality implemented. And indeed if I dump the run_output.yaml
, I can clearly see that the information is available under run.error
.
But why don't I see any error messages in the JSON then?
`{"math/run": {}, "binarytrees/run": {}, "fannkuchredux/run": {}, "conways_game_of_life/run": {}, "BigTensorProduct/run": {}, ...`
It appears that the reporter only has the data from the StatsHelper
. And that in turn is filled by run_processor.py
, which does something strange: It takes result.error
and mangles it somehow into some recorded_error
that the StatsHelper
can use, through its add_error
method. I see self.stats_helper.add_error(id, result.recorded_error)
here:
https://github.com/parttimenerd/temci/blob/a8d78cb52c248f1ae3f2469bbd0916b14ac9ea84/temci/run/run_processor.py#L328
But I don't think that result.recorded_error
is ever set! So all we get are strange empty entries in the velcom output.
Do you the velcom reporter?
Not sure. Do I?