ReBench
ReBench copied to clipboard
[T.1.1] Add Support for Custom Data Sources and Provenance Tracking
It would be useful to be able to track custom metrics, for instance:
- [ ] hardware counters
- [ ] memory usage
- [x] specific parts of a program's execution
- [ ] any kind of custom metrics
- [ ] single value metrics for builds, run jobs (overall time, binary size, ...)
ReBench can already capture such data from a single data source, it does not yet support reading data from different input sources, and it doesn't yet have a mechanism to get single values, from the "build", which are not necessarily associated with a specific run.
Having different data sources/files is pretty common, especially when using tools to measure metrics.
The challenge is to keep simple things simple, and make complex customization require less effort than writing a custom tool. This is key to make ReBench useful for a wider community.
This needs to be applied to the tracking of custom provenance information, too. For instance, tracking of GPU drivers versions, processor temperature, and CPU microcode revisions.