qiskit
qiskit copied to clipboard
more useful output for update_fake_backends.py
This PR extends tools/update_fake_backends.py to warning the user when the snapshot has errors of 1.
@nonhermitian noticed that some calibration data was bad when I was creating FakeGeneva in https://github.com/Qiskit/qiskit-terra/pull/8322#issuecomment-1181595235
Indeed, many property json files contain gates with gate_error==1
$ rgrep '"gate_error", "unit": "", "value": 1}' qiskit -l
qiskit/providers/fake_provider/backends/rochester/props_rochester.json
qiskit/providers/fake_provider/backends/washington/props_washington.json
qiskit/providers/fake_provider/backends/cambridge/props_cambridge_alt.json
qiskit/providers/fake_provider/backends/cambridge/props_cambridge.json
qiskit/providers/fake_provider/backends/manhattan/props_manhattan.json
Moreover, many exploratory backends have errors==1 somewhere, but some calibrations are better than others. So it is important to show the relative amount of gates with error==1.
Additionally, the script tools/update_fake_backends.py might silently fail and it is hard to see why is not doing what it suppose to do. Take the situation of trying to update a backend you dont have access to. It runs without any output, but does not generate the expected files.
In this PR, the script outputs when some of the args.backends did not get updated or if some of the json files were skipped.
This is how it looks:
$ ./tools/update_fake_backends.py --hub <redacted> ibm_washington
ibm_washington
==============
Readout errors == 1:
* qubit 3
Gate errors == 1:
* cx 96, 109
* cx 109, 96
* cx 118, 119
* cx 3, 4
* cx 10, 9
* cx 9, 10
* cx 119, 118
* cx 4, 3
╒═══════════════╤═════════════════════════════════╤═════════════════════════════════════╕
│ error == 1 │ new 2022-07-24 09:02:23+02:00 │ current 2022-04-12 23:42:47+09:00 │
╞═══════════════╪═════════════════════════════════╪═════════════════════════════════════╡
│ readout error │ 1/127 (0.79%) │ 0/127 (0.00%) │
├───────────────┼─────────────────────────────────┼─────────────────────────────────────┤
│ cx error │ 8/284 (2.82%) │ 6/284 (2.11%) │
├───────────────┼─────────────────────────────────┼─────────────────────────────────────┤
│ all gates │ 8/919 (0.87%) │ 6/919 (0.65%) │
╘═══════════════╧═════════════════════════════════╧═════════════════════════════════════╛
In addition, I added a parameter --datetime to be able to fetch specific snapshots, like suggested here.
Thank you for opening a new pull request.
Before your PR can be merged it will first need to pass continuous integration tests and be reviewed. Sometimes the review process can be slow, so please be patient.
While you're waiting, please feel free to review other open PRs. While only a subset of people are authorized to approve pull requests for merging, everyone is encouraged to review open pull requests. Doing reviews helps reduce the burden on the core team and helps make the project's code better for everyone.
One or more of the the following people are requested to review this:
- @Qiskit/terra-core
Pull Request Test Coverage Report for Build 3146527252
- 0 of 0 changed or added relevant lines in 0 files are covered.
- 11 unchanged lines in 1 file lost coverage.
- Overall coverage increased (+0.003%) to 84.492%
| Files with Coverage Reduction | New Missed Lines | % |
|---|---|---|
| qiskit/compiler/transpiler.py | 11 | 89.71% |
| <!-- | Total: | 11 |
| Totals | |
|---|---|
| Change from base Build 3146373335: | 0.003% |
| Covered Lines: | 60199 |
| Relevant Lines: | 71248 |
💛 - Coveralls
I added some extra explanation and follow the advice in https://github.com/Qiskit/qiskit-terra/pull/8336#discussion_r947289397
ibm_washington
==============
Broken operations (gates with errors == 1)
* cx 96, 109
* cx 109, 96
* cx 118, 119
* cx 3, 4
* cx 10, 9
* cx 9, 10
* cx 119, 118
* cx 4, 3
╒═════════════════════════════════════╤═════════════════════════════════╤═════════════════════════════════════╕
│ │ new 2022-07-23 23:10:17+02:00 │ current 2022-04-11 14:24:11+02:00 │
╞═════════════════════════════════════╪═════════════════════════════════╪═════════════════════════════════════╡
│ broken cx / total cx │ 8/284 (2.82%) │ 6/284 (2.11%) │
├─────────────────────────────────────┼─────────────────────────────────┼─────────────────────────────────────┤
│ broken operations / total operation │ 8/919 (0.87%) │ 6/919 (0.65%) │
╘═════════════════════════════════════╧═════════════════════════════════╧═════════════════════════════════════╛
The calibration you are pulling (in the "new" column) contain some broken error values (when the error is 1). However, if the rate of broken errors is better than the current one, you might want to update the files.
Closing in favor of https://github.com/Qiskit/qiskit-ibm-runtime/pull/1262 as IBM specific fake backends are moving to https://github.com/Qiskit/qiskit-ibm-runtime