HotShot icon indicating copy to clipboard operation
HotShot copied to clipboard

[Auto benchmarks] - Allow for multiple runs

Open rob-maron opened this issue 1 year ago • 0 comments

What is this task and why do we need to work on it?

For benchmarking, we have the need for running multiple runs either sequentially or concurrently. This allows us to collect data with lower variance and higher confidence.

What work will need to be done to complete this task?

This task likely involves the following:

  • [ ] A script to run the same example multiple times
  • [ ] Integrating that into the CI (when it's done)

Are there any other details to include?

No response

What are the acceptance criteria to close this issue?

A script that can run the examples multiple times on the same config file multiple times.

Branch work will be merged to (if not the default branch)

No response

rob-maron avatar Jan 08 '24 20:01 rob-maron