feat: Parallel exection of prompts in Prompt Studio

[hari-kuriakose]

Is your feature request related to a problem? Please describe. Currently prompts defined in a Prompt Studio are executed sequentially by default. This drastically increases the latency as the number of defined prompts increases.

Describe the solution you'd like

• The defined prompts need to be executed in parallel instead for obvious latency improvements.
• Might be better to add a toggle button to choose whether to run all prompts in parallel or not.

Describe alternatives you've considered

• SinglePassExtraction might not be feasible if expected output and/or passed context is big for each prompt.
• SummarizedExtraction might not be feasible if each clause needs to be analyzed (e.g. insurance contracts documents).

Additional context Credits to Michel Medawar. See original public Slack thread.

[satya9399]

I have same issue