open-interpreter
open-interpreter copied to clipboard
OpenInterpreter
Interpreter hangs on macOS when executing any shell command
Describe the bug
Interpreter always hangs on macOS when executing shell commands which makes it practically unusable as it disables both shell commands and apple script. Bug may be the same as #556 or #557 , but as both are old, opening a new one. Tested with different shells and different versions of Python on macOS.
Reproduce
On macOS 13.6.2 Ventura running on an M1 MacBook Pro:
- Install python via brew (tested with 3.7 and 3.12)
- Install open-interpreter with pip
- Configure interpreter to use gpt-4-turbo and run "interpreter -y"
- Ask "list the files in my home directory." and confirm
- Interpreter attempts to execute "ls ~" but hangs
- CTRL-C aborts the attempt, error below.
Expected behavior
No hanging.
Screenshots
list the files in my home directory.
Plan:
1 Execute a shell command to list the files in your home directory.
Now, I will execute the shell command to list the files.
Do it.
[EDIT: At this point it hangs, I have to interrupw tih CTRL-C]
ls ~
Traceback (most recent call last): File "/opt/homebrew/lib/python3.11/site-packages/interpreter/core/respond.py", line 205, in respond for line in interpreter.computer.run(language, code): File "/opt/homebrew/lib/python3.11/site-packages/interpreter/core/computer/terminal/terminal.py", line 40, in run for chunk in self._active_languages.run(code): File "/opt/homebrew/lib/python3.11/site-packages/interpreter/core/computer/terminal/languages/subprocess_language. py", line 128, in run output = self.output_queue.get(timeout=0.3) # Waits for 0.3 seconds ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/[email protected]/3.11.7/Frameworks/Python.framework/Versions/3.11/lib/python3.11/queue.py", line 180, in get self.not_empty.wait(remaining) File "/opt/homebrew/Cellar/[email protected]/3.11.7/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.p y", line 331, in wait gotit = waiter.acquire(True, timeout)
Open Interpreter version
0.2.0
Python version
3.12.1 (also tested with 3.7 with identical results)
Operating System name and version
macOS 13.6.2 Ventura
Additional context
Also tested zsh, bash and fish. All shells produce the same result.
To install Open Interpreter, you’ll need Python 3.10 or 3.11. Run python --version to check your version.
https://docs.openinterpreter.com/getting-started/setup
Re-tested with Python 3.11 with identical results. Full output below.
gappenzeller@GAppenzellerMBP14 /o/h/bin (stable)> interpreter -y
show a list of the files in the /tmp directory.
ls /tmp
Traceback (most recent call last): File "/opt/homebrew/lib/python3.11/site-packages/interpreter/core/respond.py", line 205, in respond for line in interpreter.computer.run(language, code): File "/opt/homebrew/lib/python3.11/site-packages/interpreter/core/computer/terminal/terminal.py", line 40, in run for chunk in self._active_languages.run(code): File "/opt/homebrew/lib/python3.11/site-packages/interpreter/core/computer/terminal/languages/subprocess_language.py", line 128, in run output = self.output_queue.get(timeout=0.3) # Waits for 0.3 seconds ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/[email protected]/3.11.7/Frameworks/Python.framework/Versions/3.11/lib/python3.11/queue.py", line 180, in get self.not_empty.wait(remaining) File "/opt/homebrew/Cellar/[email protected]/3.11.7/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 331, in wait gotit = waiter.acquire(True, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ KeyboardInterrupt
Can you run:
interpreter --debug
and paste the output here? Make sure to remove any api-keys should they appear...
Can you run:
interpreter --debugand paste the output here? Make sure to remove any api-keys should they appear...
interpreter --debug (oi) 21:47:22
usage: interpreter [-h] [-ci CUSTOM_INSTRUCTIONS] [-s SYSTEM_MESSAGE] [-y] [-v] [-m MODEL] [-t TEMPERATURE] [-lsv]
[-lsf] [-c CONTEXT_WINDOW] [-x MAX_TOKENS] [-b MAX_BUDGET] [-ab API_BASE] [-ak API_KEY]
[-av API_VERSION] [-xo MAX_OUTPUT] [-fc] [-dt] [-o] [-sm] [-safe {off,ask,auto}] [-cf CONFIG_FILE]
[-f] [-l] [-vi] [-os] [--config] [--reset_config] [--conversations] [--version]
interpreter: error: unrecognized arguments: --debug
I have the same problem as the OP. Python=3.11.
@Notnaton @ibehnam it should be -v now: --debug_mode / -d has been renamed to --verbose / -v.
@Notnaton @ibehnam it should be
-vnow:--debug_mode/-dhas been renamed to--verbose/-v.
Yeah this showed the debug messages but all of them stop as soon as OI is asked to execute the AppleScript command. This is the last message I see (I have to CTRL-C twice to close OI; when I do CTRL-C once, I see the error message below and OI starts talking about the fact that the operation was not successful, so I must CTRL-C again.
Traceback (most recent call last):
File "/Users/behnam/opt/anaconda3/envs/oi/lib/python3.11/site-packages/interpreter/core/respond.py", line 205, in
respond
for line in interpreter.computer.run(language, code):
File
"/Users/behnam/opt/anaconda3/envs/oi/lib/python3.11/site-packages/interpreter/core/computer/terminal/terminal.py",
line 40, in run
for chunk in self._active_languages.run(code):
File
"/Users/behnam/opt/anaconda3/envs/oi/lib/python3.11/site-packages/interpreter/core/computer/terminal/languages/subpro
cess_language.py", line 128, in run
output = self.output_queue.get(timeout=0.3) # Waits for 0.3 seconds
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/behnam/opt/anaconda3/envs/oi/lib/python3.11/queue.py", line 180, in get
self.not_empty.wait(remaining)
File "/Users/behnam/opt/anaconda3/envs/oi/lib/python3.11/threading.py", line 331, in wait
gotit = waiter.acquire(True, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OI's message after this (I CTRL-C-interrupted it):
The traceback you've posted indicates that there was aKeyboardInterrupt which is...
Ran it with -v -y. Output is about 100k lines and non-trivial to scrub. Below the lines before/after the shell call. I agree with @ibehnam that there isn't much to work with here.
[...] completion obj content: It model_response: ModelResponse(id='chatcmpl-e2ef1d19-5cc9-4a65-a769-f834e7e58db2', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(content=None, role=None))], created=1705030918, model='gpt-4-1106-preview', object='chat.completion.chunk', system_fingerprint=None, usage=Usage()); completion_obj: {'content': 'It'} model_response finish reason 3: None hold - False, model_response_str - It model_response: ModelResponse(id='chatcmpl-8g2ibbmwfYeRb6HDvgxw2wm6yLNyS', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(tool_calls=None, function_call=None, content='It', role='assistant'))], created=1705030918, model='gpt-4-1106-preview', object='chat.completion.chunk', system_fingerprint='fp_168383a679', usage=Usage()) PROCESSED CHUNK POST CHUNK CREATOR: ModelResponse(id='chatcmpl-8g2ibbmwfYeRb6HDvgxw2wm6yLNyS', choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(tool_calls=None, function_call=None, content='It', role='assistant'))], created=1705030918, model='gpt-4-1106-preview', object='chat.completion.chunk', system_fingerprint='fp_168383a679', usage=Usage()) Logging Details LiteLLM-Success Call Chunk in
terminal_interface: {'role': 'computer', 'type': 'console', 'end': True} success callbacks: []ls /tmp
Traceback (most recent call last): File "/opt/homebrew/lib/python3.11/site-packages/interpreter/core/respond.py", line 205, in respond for line in interpreter.computer.run(language, code): File "/opt/homebrew/lib/python3.11/site-packages/interpreter/core/computer/terminal/terminal.py", line 40, in run for chunk in self._active_languages.run(code): File "/opt/homebrew/lib/python3.11/site-packages/interpreter/core/computer/terminal/languages/subprocess_language.py", line 128, in run output = self.output_queue.get(timeout=0.3) # Waits for 0.3 seconds ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/[email protected]/3.11.7/Frameworks/Python.framework/Versions/3.11/lib/python3.11/queue.py", line 180, in get self.not_empty.wait(remaining) File "/opt/homebrew/Cellar/[email protected]/3.11.7/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 331, in wait gotit = waiter.acquire(True, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ KeyboardInterrupt
Chunk in
terminal_interface: {'role': 'assistant', 'type': 'message', 'start': True} Chunk interminal_interface: {'role': 'assistant', 'type': 'message', 'content': 'It'} value of chunk: ChatCompletionChunk(id='chatcmpl-8g2ibbmwfYeRb6HDvgxw2wm6yLNyS', choices=[Choice(delta=ChoiceDelta(content=' seems', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0, logprobs=None)], created=1705030917, model='gpt-4-1106-preview', object='chat.completion.chunk', system_fingerprint='fp_168383a679') PROCESSED CHUNK PRE CHUNK CREATOR: ChatCompletionChunk(id='chatcmpl-8g2ibbmwfYeRb6HDvgxw2wm6yLNyS', choices=[Choice(delta=ChoiceDelta(content=' seems', function_call=None, role=None, tool_calls=None), finish_reason=None, index=0, logprobs=None)], created=1705030917, model='gpt-4-1106-preview', object='chat.completion.chunk', system_fingerprint='fp_168383a679')
I couldn't get the shell to work, but sudo interpreter got it working, just FYI.
I couldn't get the shell to work, but
sudo interpretergot it working, just FYI.
adding sudo doesn't fix it for me, hangs on any command execution as well. ipython and python code execution works.
With some tests I found that when OI try to execute some sudo command, the password-input prompt from the OS may be removed unexpectedly from the context interface, which makes it looks like hanging there, however if you just input your password, it will execute. This issue doesn't occurs when starting OI with sudo interpreter because you have already permitted the process of OI run in sudo mode and all commands run by OI are actually its sub-process, then there's no need to ask for password again. I think the problem we should focus is why the password-input prompt from the OS was removed unexpectedly from the command line interface, I think this is a bug probably.
@Steve235lab I've never used to sudo with interpreter, thank you for the information and contribution!
I ran into the same or a related issue. I have some hosts in my environment set up for ssh keyed access i.e. no password required. I wanted to use OI to execute a series of commands on those hosts using ssh but I got the same hang. To diagnose the issue I used the simplest possible command i.e. 'ssh user@myhost "echo Hello!"' If I execute this in the terminal it works as expected - ssh just exits after the remote system returned "Hello!" But executing this in OI results in the hang. The issue here is that the subprocess keeps running because stdin is open and the remote host, therefore, waits for further input. I experimented with closing stdin immediately after sending the code (in core/computer/terminal/languages/subprocess.py and then: poll for process exit, make sure to read to EOF for stdout and stderr. This works as expected. The problem is that this will only work properly for interactive code which waits for additional input once it start up in the subprocess. Not really sure that this is even desirable. In any event, in case it's useful, I have attached my changes to subprocess.py that works very well for me. Perhaps there's some thought needed for making a distinction between running shell commands for interactive code versus non-interactive? Here's the updated code:
def terminate(self):
if self.process:
self.process.terminate()
self.process.stdout.close()
self.process = None
def start_process(self):
if self.process:
self.terminate()
my_env = os.environ.copy()
my_env["PYTHONIOENCODING"] = "utf-8"
self.process = subprocess.Popen(
self.start_cmd,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
bufsize=0,
universal_newlines=True,
env=my_env,
encoding="utf-8",
errors="replace"
)
threading.Thread(
target=self.handle_stream_output,
args=(self.process.stdout, False),
daemon=True,
).start()
threading.Thread(
target=self.handle_stream_output,
args=(self.process.stderr, True),
daemon=True,
).start()
def run(self, code):
try:
"""
Always start a new process. Once we close stdin
we can't send more code to the process.
"""
self.start_process()
self.process.stdin.write(code + "\n")
"""
Close stdin after sending the command.
This allows the subprocess to exist when the
code is done and stdout, stderr have been
fully read elsewhere in the code.
"""
self.process.stdin.close()
"""
Check for process termination and full
drain of output.
"""
while True:
if self.process.poll() is not None:
break
if not self.output_queue.empty():
yield self.output_queue.get()
else:
time.sleep(0.1)
# Drain any remaining output
try:
while True:
output = self.output_queue.get(timeout=0.3)
yield output
except queue.Empty:
pass
except Exception as e:
yield {
"type": "console",
"format": "output",
"content": f"Error: {str(e)}"
}
When oi runs a sudo command, I usually do not see the first password prompt. If it hangs after sudo, type in your password. Or if you want to make sure, just hit enter and it will re-prompt you for sudo password.
