Qiskit
Parallelization with Aer not working
Informations
- Qiskit Aer version: 0.8.2
- Python version: Python 3.8.10
-
Operating system:
NAME="Pop!_OS" VERSION="20.10" ID=pop ID_LIKE="ubuntu debian" PRETTY_NAME="Pop!_OS 20.10" VERSION_ID="20.10" HOME_URL="https://pop.system76.com" VERSION_CODENAME=groovy UBUNTU_CODENAME=groovy
What is the current behavior?
Calls to run a circuit in parallel hang in certain situations. If I'm using multiprocessing, and I try to simulate circuits by using a Pool, then normally everything works fine. However, if I run a single simulation outside of this pool, then any runs inside the pool will hang indefinitely.
Steps to reproduce the problem
from qiskit import QuantumCircuit
from qiskit.providers.aer import Aer
from multiprocessing import Pool
def parallel(circ2):
simulator2 = Aer.get_backend('aer_simulator')
simulator2.run(circ2).result()
print('res_2')
circ1 = QuantumCircuit(2, 2)
simulator1 = Aer.get_backend('aer_simulator')
simulator1.run(circ1).result() #comment this out to fix
print('res_1')
circs_arr = [QuantumCircuit(2, 2) for i in range (5)]
with Pool(5) as p:
p.map(parallel,circs_arr)
What is the expected behavior?
That the line simulator2.run(circ2).result() will not run indefnitely, i.e. both "res_1" and "res_2" will be printed in the above example.
Suggested solutions
I've also tried using the ray and multiprocess (which is different from multiprocessing) libraries, but both resulted in the same issue.
I've seen some discussion about this problem in the past, where some have suggested disabling Aer's internal paralellization. I've tried this, but my problem was not resolved.
I'm not sure what is your problem in the above example because it does not work in my environment (simulator2 and circ2 are not defined when simulator2.run(circ2).result() is evaluated).
On the other hand, the below codes work fine:
from qiskit import QuantumCircuit
from qiskit.providers.aer import Aer
from multiprocessing import Pool
def parallel(circ):
simulator2 = Aer.get_backend('aer_simulator')
ret2 = simulator2.run(circ).result()
print('ret2: ', ret2)
def run_program():
circ1 = QuantumCircuit(2, 2)
simulator1 = Aer.get_backend('aer_simulator')
ret1 = simulator1.run(circ1).result()
print('ret1: ', ret1)
circs_arr = [QuantumCircuit(2, 2) for i in range (5)]
with Pool(5) as p:
p.map(parallel,circs_arr)
if __name__ == '__main__':
run_program()
Anyway, basically, Aer manages parallelization in C++. I do not recommend to parallelize simulation within python side(ex: with multiprocessing.Pool).
@hhorii Thanks for the reply. I can see that I accidentally included a line that shouldn't have been there, but I've fixed it. The posted code should now demonstrate my problem. I've tried running the code you provided, and it doesn't look like it worked- I've encountered the same issue that I've been having. It's interesting that your code works fine on your machine. Which operating system are you running it on?
I'm sorry that I have not answered to your question for a long time. I believe that this issue is not happened with 0.11 on MacOS (12.6) and python 3.9. Please reopen this if you still have troubles and need to fix it.