Cirq icon indicating copy to clipboard operation
Cirq copied to clipboard

Improve performance benchmarks coverage after migration to ASV

Open tanujkhattar opened this issue 4 years ago • 1 comments

Currently, the performance benchmarks in Cirq are very limited. We should improve the coverage of performance benchmarks by adding more tests.

We should discuss

  • Which functions / code pieces are worth benchmarking?
  • Should we have benchmarks for all functions / modules ?

Decide a strategy and then implement the additional benchmarks using ASV.

Related: #3822 #1124

tanujkhattar avatar Feb 19 '21 22:02 tanujkhattar

Discussed in Cirq sync:

  • We need to incorporate benchmarks into the PR review workflow (specifically highlighting regressions, not blocking PRs)
  • General concern around hard-to-detect gradual slowdown of code

@tanujkhattar Do we have this kind of support, or designs around it?

95-martin-orion avatar Apr 28 '21 18:04 95-martin-orion