[ NEW PAGE ] Benchmarking performance

[javabster]

With performance becoming a big feature of qiskit for the rest of the year we should include a page somewhere in docs (not sure where yet) that explains the following:

• what do we mean by "performance" (e.g. speed of transpilation, memory usage, number of cnot gates). Discuss challenges of quantum performance metrics and why they change often
• why are these metrics important? why should I, a quantum developer, care about these? what value does it add to my quantum program?
• where can I find benchmarking information, or how could I run the benchmarks myself?
• lists of papers relevant to each metric
• small section in latest update page about latest performance improvements?

~Options for where this content could live:~ ~- the "additional resources" tab~ ~- the transpilation specific information could live in the transpile section, not sure about the other information~ ~- if there are multiple pages that could be created maybe it's worth creating a "performance" or "benchmarking" folder in the guides section? Might be confusing though if we also want to put performance discussion interweaved with the other sections on error mitigation or transpilation thought~

update: put "performance" folder in guides section, with at least 2 pages (1x SDK performance, 1x hardware/other performance)

questions to discuss: ~- do we just want to talk about the software performance (e.g. transpile time) or also hardware (quantum volume, CLOPS etc.)~

• how much is SDK specific vs runtime vs transpiler service

[sbrandhsn]

I like this! :-) We should probably also argue why we think the particular benchmark selection is reasonable. I think we should also have a discussion outlining a rust vs python discussion and the associated advantages/drawbacks.