llm-workflow-engine

llm-workflow-engine copied to clipboard

Say I want to receive 1000 responses and store them somewhere. I have a ChatGPT Plus subscription. Is there a way I can make multiple queries at once (such as with multiple accounts, having both Firefox and Chromium open, etc.) to speed up this process?

This should be in theory possible (if openai allows it). Playwright allows you to start multiple firefox instances at the same time, they just need to have different users. Implementing it (in a non hacky way) is non-trivial though and I don't have a plus subscription to test :) I'd welcome any proposal on how to deal with that and a PR

I pushed an update that fixes the issue. Now it should be possible to open multiple session of chatgpt shell (or through python api) at the same time. Not sure however if openai allows to run 1000 queries at the same time.

Please let me know if it works for you and what are the maximum number of queries you can run in parallel

I'm not sure if I'm using your update properly, but when I have 2 terminals open and run my query script (main.py) in each at the same time, one will return me the responses as I expect, the other will print this:

(ChatGPTWrapper) maxreuter@Maxs-Mac-mini chatgpt-wrapper % python main.py
Traceback (most recent call last):
  File "/Users/maxreuter/Documents/Personal projects/chatgpt-wrapper/chatgpt_wrapper/chatgpt.py", line 39, in __init__
    proxy=proxy,
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/sync_api/_generated.py", line 14711, in launch_persistent_context
    recordHarContent=record_har_content,
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/_impl/_sync_base.py", line 104, in _sync
    return task.result()
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/_impl/_browser_type.py", line 155, in launch_persistent_context
    from_channel(await self._channel.send("launchPersistentContext", params)),
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/_impl/_connection.py", line 45, in send
    lambda: self.inner_send(method, params, False)
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/_impl/_connection.py", line 419, in wrap_api_call
    return await cb()
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/_impl/_connection.py", line 79, in inner_send
    result = next(iter(done)).result()
playwright._impl._api_types.Error: Browser.enable): Browser closed.
==================== Browser output: ====================
<launching> /Users/maxreuter/Library/Caches/ms-playwright/firefox-1372/firefox/Nightly.app/Contents/MacOS/firefox -no-remote -headless -profile /tmp/playwright -juggler-pipe about:blank
<launched> pid=51177
[pid=51177][err] *** You are running in headless mode.
[pid=51177][err] A copy of Firefox is already open. Only one copy of Firefox can be open at a time.
[pid=51177] <process did exit: exitCode=0, signal=null>
[pid=51177] starting temporary directories cleanup
=========================== logs ===========================
<launching> /Users/maxreuter/Library/Caches/ms-playwright/firefox-1372/firefox/Nightly.app/Contents/MacOS/firefox -no-remote -headless -profile /tmp/playwright -juggler-pipe about:blank
<launched> pid=51177
[pid=51177][err] *** You are running in headless mode.
[pid=51177][err] A copy of Firefox is already open. Only one copy of Firefox can be open at a time.
[pid=51177] <process did exit: exitCode=0, signal=null>
[pid=51177] starting temporary directories cleanup
============================================================

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 5, in <module>
    bot = ChatGPT()
  File "/Users/maxreuter/Documents/Personal projects/chatgpt-wrapper/chatgpt_wrapper/chatgpt.py", line 43, in __init__
    shutil.copytree("/tmp/playwright", self.user_data_dir)
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/shutil.py", line 368, in copytree
    raise Error(errors)
shutil.Error: [('/tmp/playwright/IndexedDB/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0', '/tmp/67d0b65a-214a-4a2d-9afe-21cf36220fbe/IndexedDB/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0', "[Errno 62] Too many levels of symbolic links: '/tmp/playwright/IndexedDB/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0/v0'")]

hmm strange. Can you remove the folder /tmp/playwright and uninstall and install the package and try again. First doing chatgpt install Then running chatgpt and testing a prompt Then running a second chatgpt shell and testing another prompt

Ok, I did that. At first, when I ran my query script again, I got this message:

(ChatGPTWrapper) maxreuter@Maxs-Mac-mini chatgpt-wrapper % python main.py
Traceback (most recent call last):
  File "main.py", line 5, in <module>
    bot = ChatGPT()
  File "/Users/maxreuter/Documents/Personal projects/chatgpt-wrapper/chatgpt_wrapper/chatgpt.py", line 54, in __init__
    self._start_browser()
  File "/Users/maxreuter/Documents/Personal projects/chatgpt-wrapper/chatgpt_wrapper/chatgpt.py", line 62, in _start_browser
    self.page.goto("https://chat.openai.com/")
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/sync_api/_generated.py", line 9185, in goto
    url=url, timeout=timeout, waitUntil=wait_until, referer=referer
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/_impl/_sync_base.py", line 104, in _sync
    return task.result()
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/_impl/_page.py", line 491, in goto
    return await self._main_frame.goto(**locals_to_params(locals()))
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/_impl/_frame.py", line 147, in goto
    await self._channel.send("goto", locals_to_params(locals()))
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/_impl/_connection.py", line 45, in send
    lambda: self.inner_send(method, params, False)
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/_impl/_connection.py", line 419, in wrap_api_call
    return await cb()
  File "/Users/maxreuter/opt/miniconda3/envs/ChatGPTWrapper/lib/python3.7/site-packages/playwright/_impl/_connection.py", line 79, in inner_send
    result = next(iter(done)).result()
playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.
=========================== logs ===========================
navigating to "https://chat.openai.com/", waiting until "load"
============================================================

I tried again, and it looks like the requests are running sequentially, and sometimes "collide" with each other (see outputs below). It seems they're competing for a resource that can be owned by only 1 of them at a time. However, it looks like the first run seemed to work out mostly okay -- not sure what to make of that.

Here's 5 runs when I ran the script at the same time in each terminal.

Terminal 1 output:

(ChatGPTWrapper) maxreuter@Maxs-Mac-mini chatgpt-wrapper % python main.py
["Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'", '1+6 = 7.', '1+6 = 7.', '2+5 = 7.', '2+4 = 6.']
Time elapsed: 6.0182929039001465

(ChatGPTWrapper) maxreuter@Maxs-Mac-mini chatgpt-wrapper % python main.py
['The sum of 1 and 5 is 6.', 'The sum of 3 and 6 is 9.', 'The sum of 3 and 6 is 9.', 'The sum of 3 and 4 is 7.', 'The sum of 3 and 6 is 9.']
Time elapsed: 11.002346992492676

(ChatGPTWrapper) maxreuter@Maxs-Mac-mini chatgpt-wrapper % python main.py
['1+5 is 6.', '3+6 is 9.', '1+5 is 6.', '1+6 is 7.', '1+6 is 7.']
Time elapsed: 7.22987699508667

(ChatGPTWrapper) maxreuter@Maxs-Mac-mini chatgpt-wrapper % python main.py
['The sum of 3 and 5 is 8.', 'The sum of 2 and 6 is 8.', 'The sum of 2 and 6 is 8.', 'The sum of 2 and 5 is 7.', "Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'"]
Time elapsed: 8.695698022842407

(ChatGPTWrapper) maxreuter@Maxs-Mac-mini chatgpt-wrapper % python main.py
['The sum of 3 and 4 is 7.', 'The sum of 3 and 5 is 8.', 'The sum of 2 and 4 is 6.', 'The sum of 1 and 4 is 5.', 'The sum of 3 and 6 is 9.']
Time elapsed: 10.052808046340942

Terminal 2 output:

(ChatGPTWrapper) maxreuter@Maxs-Mac-mini chatgpt-wrapper % python main.py
['1+5 equals 6.', '1+4 equals 5.', '3+5 equals 8.', '2+5 equals 7.', '2+4 equals 6.']
Time elapsed: 6.820550203323364

(ChatGPTWrapper) maxreuter@Maxs-Mac-mini chatgpt-wrapper % python main.py
["Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'", "Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'", "Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'", "Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'", "Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'"]
Time elapsed: 8.518801927566528

(ChatGPTWrapper) maxreuter@Maxs-Mac-mini chatgpt-wrapper % python main.py
['The sum of 1 and 4 is 5.', 'The sum of 2 and 4 is 6.', 'The sum of 2 and 4 is 6.', 'The sum of 1 and 6 is 7.', 'The sum of 3 and 5 is 8.']
Time elapsed: 16.01532006263733

(ChatGPTWrapper) maxreuter@Maxs-Mac-mini chatgpt-wrapper % python main.py
["Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'", "Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'", "Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'", '2+5 is equal to 7.', '1+4 is equal to 5.']
Time elapsed: 10.773230075836182

(ChatGPTWrapper) maxreuter@Maxs-Mac-mini chatgpt-wrapper % python main.py
["Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'", "Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'", "Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'", "Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'", "Unusable response produced, maybe login session expired. Try 'pkill firefox' and 'chatgpt install'"]
Time elapsed: 8.572566032409668

what happens when you run multiple requests at the same time in multiple views of the webapp? Is it possible or do you get an error. I know that for the free account you cannot have more than one request running at the same time

Browser backend is deprecated, and support for this kind of mass usage is not supported. Suggest you use the API backend, or the OpenAI endpoints directly.