ocurrent
Do not run by default on all PRs
A very valid concern for the benchmarks is the energy cost it takes to run them, and all PRs do not need benchmarks. A nice thing to do can be perhaps have a github bot (I have seen these on Rust repositories), and when it's invoked only then we run the benchmarks. I haven't given this a lot of thought, there are perhaps better ways to do this. :)
Are you suggesting that we make the benchmark runs triggered manually, instead of automatically running on every PR?
Yes! I don't know if it's a good idea, but IIRC maybe rust does this where you can invoke a bot for running benchmarks on the PR? But we can run the benchmarks everyday on the default branch? I don't have the idea very fleshed out, but something to think about :)