No benefit from using advanced resilience_levels in Estimator

[nonhermitian]

The IBM Runtime has resilience_levels that controls how much error mitigation goes into a result. In general, it is supposed that the higher the resilience_level the more accurate the resulting expectation values are. However, this does not appear to be the case.

Consider the circuit:

N = 6
qc2 = QuantumCircuit(N)

qc2.x(range(N))
qc2.h(range(N))

for kk in range(N // 2, 0, -1):
    qc2.ch(kk, kk - 1)
for kk in range(N // 2, N - 1):
    qc2.ch(kk, kk + 1)

and operator:

oper = SparsePauliOp.from_list([('Z'*6, 1)])

Running through all the possible resilience_levels on ibm_geneva using:

with Session(backend=backend):
    estimator = Estimator()
    job0 = estimator.run(qc2, oper, shots=1e4, resilience_level=0)
    job1 = estimator.run(qc2, oper, shots=1e4, resilience_level=1)
    job2 = estimator.run(qc2, oper, shots=1e4, resilience_level=2)
    job3 = estimator.run(qc2, oper, shots=1e4, resilience_level=3)

gives expectation values [0.203, 0.4525, 0.2165, 0.1497], respectively, where the ideal value is 0.4374. Thus level 1 (TREX) overshoots the answer (over-corrects), level 2 (ZNE) does very little (within shot variance I think), and level 3 (PEC) makes things worse.

[blakejohnson]

The drop from resilience level 1 to 2 is consistent with the fact that you have a short depth circuit dominated by measurement error. We really need level 2 to do both TREX and ZNE.

The failure of PEC in this case at level 3 deserves some standalone debug. How long did job3 take, @nonhermitian?

[nonhermitian]

Ok yeah. I assumed that TREX was being done at level 2 since it is a straightforward addition.

The PEC job took about 43min.

[nonhermitian]

For the PEC issues I will add that mitigating simple circuits like 4Q GHZ states is markedly off. In this case the actual should be 1.0 but PEC returns a much larger number. Also note that standard_error seems to be erroring somewhere:

EstimatorResult(values=array([1.72611795]), metadata=[{'standard_error': nan, 'confidence_interval': [1.6989553989420931, 1.753280495527756], 'confidence_level': 0.95, 'shots': 1498496, 'samples': 11707, 'sampling_overhead': 1.1707446507600376, 'total_mitigated_layers': 2}])

Job time was 30 min

[kt474]

@nonhermitian Have you experienced similar results when using EstimatorV2?

[kt474]

Was not able to replicate this with EstimatorV2, closing for now. Feel free to reopen if you are still experiencing this issue.