Request to Allow multiple runs in parallel for the Autotuner

[dalyles]

Description

I'm trying to save time by using the autotuner to run multiple circuits at once. At the moment I can only run one circuit at a time.

This is what comes out when I attempt to do a second run on a different circuit while another run is in progress

Suggested Solution

Is it possible to enhance the distributed.py script script such that running multiple jobs in parallel is possible? This would save significant time when there's a need to run multiple circuits on demand.

Thank you.

Additional Context

No response

[vvbandeira]

Please check python3 distributed.py tune -h; it will give you knobs to set the number of jobs, samples, iterations and resources. From your question, the ones you should focus on are --jobs and --resources_per_trial.

[dalyles]

Hi @vvbandeira

What I mean to say is lets say I have cktA and cktB, they are both separate circuits and I have to run two separate commnds for them. I'd like both commands to be executed succesfully without having one of them get killed. That is what is demonstrated in the screenshot.

[vvbandeira]

Would the runs be linked/dependent on each other in any way? Or do you want a wrapper script that does the equivalent of:

python3 distributed.py --design cktA tune &
python3 distributed.py --design cktB tune &
wait

I am not sure if I understand the request.

[dalyles]

The runs are independent from one another. Completely separate from another. I'm trying to run cktA, cktB, cktC etc in completely separate command line windows, and the expected behavior is that they all can execute. What is happening is that the Autotuner as of now can not take a command when it is currently executing a previous command (e.g. cannot run autotuner on cktB when cktA is still running). I have to wait for cktA's run to terminate before I can run cktB.

I hope that makes sense.

As for a wrapper script like the one you put, Id like to execute that, does it generate an error when you try to run two or multiple circuits in parallel with this setup?

Thank you

[dalyles]

Just wanted to follow up and see if there is an interest in this enhancement request.

Thank you :)

[vvbandeira]

@dalyles The wrapper option would work. The only caveat is that you need to take into account the amount of resources your machine/cloud has available -- since each process does not know about the other.

The more sophisticated approach, running both circuits with a single call of ./distributed.py, would require some planning and development time. At the moment, we do not have the bandwidth to tackle this project. If you are interested in doing the work and contributing to the project, that would be wonderful. We are happy to help in any way we can.