ReBench icon indicating copy to clipboard operation
ReBench copied to clipboard

[T.1.1] Add Support for Custom Data Sources and Provenance Tracking

Open smarr opened this issue 4 years ago • 0 comments

It would be useful to be able to track custom metrics, for instance:

  • [ ] hardware counters
  • [ ] memory usage
  • [x] specific parts of a program's execution
  • [ ] any kind of custom metrics
  • [ ] single value metrics for builds, run jobs (overall time, binary size, ...)

ReBench can already capture such data from a single data source, it does not yet support reading data from different input sources, and it doesn't yet have a mechanism to get single values, from the "build", which are not necessarily associated with a specific run.

Having different data sources/files is pretty common, especially when using tools to measure metrics.

The challenge is to keep simple things simple, and make complex customization require less effort than writing a custom tool. This is key to make ReBench useful for a wider community.

This needs to be applied to the tracking of custom provenance information, too. For instance, tracking of GPU drivers versions, processor temperature, and CPU microcode revisions.

smarr avatar Dec 04 '21 11:12 smarr