HotShot
HotShot copied to clipboard
[Auto benchmarks] - Allow for multiple runs
What is this task and why do we need to work on it?
For benchmarking, we have the need for running multiple runs either sequentially or concurrently. This allows us to collect data with lower variance and higher confidence.
What work will need to be done to complete this task?
This task likely involves the following:
- [ ] A script to run the same example multiple times
- [ ] Integrating that into the CI (when it's done)
Are there any other details to include?
No response
What are the acceptance criteria to close this issue?
A script that can run the examples multiple times on the same config file multiple times.
Branch work will be merged to (if not the default branch)
No response