Stefan Marr
Stefan Marr
Expand ReBenchDB's support for data provenance by detecting environmental differences that affect the comparability of results. Operating system, software, or CPU microcode updates can change performance and make comparison with...
Optimize result storage to increase scalability for ReBenchDB to reduce memory and storage requirements. Hopefully, we can reduce the system's requirements, allow a larger number of active users/requests, and make...
Setup a staging environment for ReBenchDB, which we deploy to automatically from GitHub.
In some scenarios, one would want to run benchmarks not one the local machine, but perhaps some extension device, or a remote devices that is not able to run ReBench....
ReBench already configures benchmarking machines to minimize measurement error. The [Krun benchmark runner](https://github.com/softdevteam/krun} supports additional configuration settings and operating systems. This includes active thermal management, additional setting validation, and perturbation...
It would be useful to be able to track custom metrics, for instance: - [ ] hardware counters - [ ] memory usage - [x] specific parts of a program's...
Looking at changes in benchmark numbers is unfortunately rarely very insightful by itself. To understand what benchmarks spend their time on, it would be useful to add support for profiling....
In #170, we added the ability to mark as specific as important, to break out of the normal resolution priority. Though, the PR didn't document the feature yet. Needs to...
At the moment, we compose experiments/runIs bottom up, with the highest elements having priority. See https://rebench.readthedocs.io/en/latest/config/#priority-of-configuration-elements Though, in some cases this doesn't seem to be very useful. For instance, if...
Currently denoise sets settings as it desires, and then guesses default settings, but does not actually use the previous system settings when restoring them. This seems less than ideal. We...