Fake backends ignore errors related to basis gates or number of qubits

[yaelbh]

Environment

• Qiskit Terra version: 0.20.0
• Python version: 3.9.5
• Operating system: Linux

What is happening?

Fake backends' behavior is sometimes different from their corresponding real devices.

Concretely, a device won't run a circuit that contains gates that do not belong to the set of basis gates. The fake backend will run the circuit.

Similarly, the device won't run a circuit that refers to non-existing qubits, while the fake backend will.

How can we reproduce the issue?

Code:

from qiskit import QuantumCircuit, IBMQ
from qiskit.providers.fake_provider import FakeLima
from qiskit.providers.backend import BackendV1

provider = IBMQ.load_account()
backends = [provider.backend.ibmq_lima, FakeLima()]

circ_not_a_basis_gate = QuantumCircuit(1, 1)
circ_not_a_basis_gate.h(0)
circ_not_a_basis_gate.measure(0, 0)
circ_not_a_basis_gate.name = "not-a-basis-gate"

circ_many_qubits = QuantumCircuit(10, 10)
circ_many_qubits.measure(9, 9)
circ_many_qubits.name = "many-qubits"

for circ in [circ_not_a_basis_gate, circ_many_qubits]:
    for backend in backends:
        print(f"\nRunning the {circ.name} circuit on {backend}:")

        job = backend.run(circ)

        try:
            print("Job result:", job.result())
        except Exception as e:
            print("Exception when trying to fetch job result:", e)

        print("Job status:", job.status())

Output:

Running the not-a-basis-gate circuit on ibmq_lima:
Exception when trying to fetch job result: "Unable to retrieve result for job 62f35351eace5881b6c259f4. Job has failed: The Qobj uses gates (['h']) that are not among the basis gates (['id', 'rz', 'sx', 'x', 'cx', 'reset']). Error code: 1106."
Job status: JobStatus.ERROR

Running the not-a-basis-gate circuit on fake_lima:
Job result: Result(backend_name='aer_simulator', backend_version='0.10.4', qobj_id='2b239bdf-2b05-49c9-9afe-e94e3175a530', job_id='2855837a-9efc-4659-8391-d2295e51e31a', success=True, results=[ExperimentResult(shots=1024, success=True, meas_level=2, data=ExperimentResultData(counts={'0x0': 525, '0x1': 499}), header=QobjExperimentHeader(clbit_labels=[['c', 0]], creg_sizes=[['c', 1]], global_phase=0.0, memory_slots=1, metadata={}, n_qubits=1, name='not-a-basis-gate', qreg_sizes=[['q', 1]], qubit_labels=[['q', 0]]), status=DONE, seed_simulator=3721132851, metadata={'parallel_state_update': 1, 'noise': 'superop', 'batched_shots_optimization': False, 'measure_sampling': True, 'device': 'CPU', 'num_qubits': 1, 'parallel_shots': 1, 'remapped_qubits': False, 'method': 'density_matrix', 'active_input_qubits': [0], 'num_clbits': 1, 'input_qubit_map': [[0, 0]], 'fusion': {'applied': False, 'max_fused_qubits': 5, 'enabled': True, 'threshold': 14}}, time_taken=0.004068663)], date=2022-08-10T09:42:30.224142, status=COMPLETED, header=QobjHeader(backend_name='aer_simulator', backend_version='0.10.4'), metadata={'time_taken': 0.019088631, 'time_taken_execute': 0.008525004, 'parallel_experiments': 1, 'omp_enabled': True, 'max_gpu_memory_mb': 0, 'num_mpi_processes': 1, 'time_taken_load_qobj': 0.010551124, 'max_memory_mb': 7068, 'mpi_rank': 0}, time_taken=0.01973700523376465)
Job status: JobStatus.DONE

Running the many-qubits circuit on ibmq_lima:
Exception when trying to fetch job result: 'Unable to retrieve result for job 62f353565838303af5f1df08. Job has failed: The number of qubits in the Qobj (10) is higher than the number of qubits supported by the device (5). Error code: 1109.'
Job status: JobStatus.ERROR

Running the many-qubits circuit on fake_lima:
Job result: Result(backend_name='aer_simulator', backend_version='0.10.4', qobj_id='bb0e3ece-bf05-4dae-bf46-5eff578199e0', job_id='8aa2036f-6d82-4c53-bf79-23a1d549fff5', success=True, results=[ExperimentResult(shots=1024, success=True, meas_level=2, data=ExperimentResultData(counts={'0x0': 1024}), header=QobjExperimentHeader(clbit_labels=[['c', 0], ['c', 1], ['c', 2], ['c', 3], ['c', 4], ['c', 5], ['c', 6], ['c', 7], ['c', 8], ['c', 9]], creg_sizes=[['c', 10]], global_phase=0.0, memory_slots=10, metadata={}, n_qubits=10, name='many-qubits', qreg_sizes=[['q', 10]], qubit_labels=[['q', 0], ['q', 1], ['q', 2], ['q', 3], ['q', 4], ['q', 5], ['q', 6], ['q', 7], ['q', 8], ['q', 9]]), status=DONE, seed_simulator=3017945355, metadata={'parallel_state_update': 1, 'noise': 'superop', 'batched_shots_optimization': False, 'measure_sampling': True, 'device': 'CPU', 'num_qubits': 1, 'parallel_shots': 1, 'remapped_qubits': True, 'method': 'density_matrix', 'active_input_qubits': [9], 'num_clbits': 10, 'input_qubit_map': [[9, 0]], 'fusion': {'applied': False, 'max_fused_qubits': 5, 'enabled': True, 'threshold': 14}}, time_taken=0.006464472)], date=2022-08-10T09:42:33.850333, status=COMPLETED, header=QobjHeader(backend_name='aer_simulator', backend_version='0.10.4'), metadata={'time_taken': 0.044194349, 'time_taken_execute': 0.013991019, 'parallel_experiments': 1, 'omp_enabled': True, 'max_gpu_memory_mb': 0, 'num_mpi_processes': 1, 'time_taken_load_qobj': 0.030191295, 'max_memory_mb': 7068, 'mpi_rank': 0}, time_taken=0.04497790336608887)
Job status: JobStatus.DONE

What should happen?

My understanding of the role of the fake backends is that they should mimic as much as possible the behavior of the device. If this view is correct, then in the code snippet above they should raise exceptions similar to the device, and set the job status to ERROR.

Any suggestions?

No response