AerSimulator results show correlation across seed numbers when using reset with the stabilizer method

[wshanks]

Informations

• Qiskit Aer version: 0.12.2
• Python version: 3.10.12
• Operating system: Linux

What is the current behavior?

When preparing the +X state and then measuring, AerSimulator gives results that are correlated across seed number when the circuit begins with a reset instruction.

I studied the behavior more thoroughly in this gist.

Steps to reproduce the problem

Besides the gist, one can look at the statistics of the counts for the following circuit as the seed number is varied. The counts should 2500 +/- sqrt(5000/4), but when looking at around 1000 consecutive seed numbers the variation is much less and the average off from 2500 by several standard deviations.

from qiskit import QuantumCircuit

from qiskit_aer import AerSimulator


circ = QuantumCircuit(1, 1)
circ.reset(0)
circ.barrier(0)
circ.barrier(0)
circ.h(0)
circ.barrier(0)
circ.measure(0, 0)

deviations = numpy.zeros(1000)
for seed in range(len(deviations)):
    backend = AerSimulator(seed_simulator=seed)
    job = backend.run(circ, shots=5000)
    deviations[seed] = job.result().get_counts()["0"] - 2500

print(numpy.std(deviations))

What is the expected behavior?

Results not correlated with seed number, with the average around 50% probability for a +X state measurement.

Suggested solutions

I am not sure, but it is possible that this behavior is related to https://github.com/Qiskit/qiskit-aer/issues/1888 and could be addressed by https://github.com/Qiskit/qiskit-aer/pull/1895 as well.

[doichanj]

I think this issue is not for stabilizer simulator but also occurs on statevector method, and it is not related to issue #1888 This issue is caused by seed number setting inside Aer, because we set the random seed for each shot as seed_simulator + shot_id. So by setting seed_simulator=seed in the script above uses same seed number for most of shots in the iteration. By setting seed_simulator=seed*5000 in the script above, we can get reasonable output.

[wshanks]

Thanks, @doichanj. Your explanation makes sense. I mentioned the stabilizer method because the ExperimentResult objects had "method": "stabilizer" in their metadata.

I defer to you on whether this effect is a bug or just expected behavior. I did an extra test and found that the seed_simulator + shot_id seed is independent of the number of circuits executed (I get the same results for the first circuit even when I add more circuits to the run() call), so the effect is not more diluted by running multiple circuits.

This effect came up for me while working on https://github.com/Qiskit-Extensions/qiskit-experiments/pull/1250. There, I wanted to remove resets from the beginning of circuits, but I found that this caused test results to change and fall outside of the test's tolerance. I think this is just because the tolerance was set too tightly. The seed effect was misleading though because I found that trying out 50 seeds that the case with reset passed every time while the case without failed about a third of the time, which seemed to indicate a significant effect. Really though it was just that the seed results were correlated for the reset case and happened to be correlated to a set of results that averaged within the test tolerance.

One other side note -- I guess without the resets that aer does a state vector simulation and then just samples from the results, while with the reset it does a sampling at the point of the reset and evolves the state separately for each shot from there? I noticed that the reset case was much slower and I am guessing this is why, and is why the case without resets is not correlated across seeds. It might be possible to catch the case of a leading reset and just ignore it in order to speed up the simulation (or catch the case where the reset is applied an unentangled state so that it is deterministic and doesn't require branche?), though perhaps it is not worth the effort.

[doichanj]

Aer automatically select a simulation method from the input circuit if user does not set method option. In your case, these are Clifford gates only so Aer set the simulation method to stabilizer. I tested both stabilizer and statevector and I got same result.

Also Aer automatically select if sampling measure can be applied or not from the input circuit. If there are some operations with randomness (reset operation in your case), sampling measure will not be used and each shot is simulated independently. In this case to parallelize simulating each shot, we set random seed as seed_simulator + shot_id to get same output when we change the number of threads to be used. (If sampling measure is enabled only seed_simulator is used for all the shots) I think we should document this behavior and user have to add seed + number of shots