Autodetected benchmark parameters should be reported in output

[ceresek]

When a benchmark autodetects a parameter (such as thread_count=$cpu.count), the actual value used should be reported in the benchmark output.

[lbulej]

Do you mean the textual output, or the JSON output file?

I was already thinking about making it possible for the writer to dump the actual configuration of a benchmark. The reason why I haven't done that yet is that I will need to change the plugin API to provide access to this information.

It would be easy to just expose the BenchmarkInfo object in the event notification, but to make it smooth, I would probably want to move the Plugin and the ExecutionPolicy classess into the harness package, so as to move the API for benchmark writers and plugin writers to separate packages.

I'll try to put this into the next API update -- these are the exact reasons I was waiting for to do something :-)

[axel22]

Perhaps we could additionally add a -v flag to enable verbose output that shows information such as this on stdout.

[ceresek]

For me the JSON output makes most sense, because that will document how the measurements were obtained, but I can imagine other use cases where one would want to see the value of the options on a console without executing the workload.

[axel22]

JSON sounds good to me then.

[farquet]

Adding it to JSON is good. But I would also add it to stdout by default at the very beginning of the execution. That's very little output and can be helpful to quickly identify differences between runs by grepping through stdouts.

Other suites do that (here dacapo) :

The derived number of threads (72) is outside the range [1,64]; rescaling to match thread limit.