All-Hands-AI
Freezes at step 0 via local model (spams ollama server)
Describe the bug
As in the title, when I send anything to OD webui, displays Starting new task... and hangs (unresponsive).
I am using ollama server and WSL with nvidia-smi, gpu usage goes to 100%, then goes to 0 after about a minute. Model loads via ollama, devin freezes on step 0 (in log), and ollama api works fine after killing all pids listening on ollama port and having ollama auto restart.
After killing ollama processes (forcing auto restart) and the connection from devin, 99 additional errors appear in the devin server log (due to 99 identical steps being created) like the following (last):
==============
STEP 99
PLAN:
🔵 0 make a python snake game
Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.
AGENT ERROR:
Error condensing thoughts: 'NoneType' object has no attribute 'request'
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/urllib3/connection.py", line 198, in _new_conn
sock = connection.create_connection(
File "/usr/local/lib/python3.10/dist-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/usr/local/lib/python3.10/dist-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py", line 793, in urlopen
response = self._make_request(
File "/usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py", line 496, in _make_request
conn.request(
File "/usr/local/lib/python3.10/dist-packages/urllib3/connection.py", line 400, in request
self.endheaders()
File "/usr/lib/python3.10/http/client.py", line 1278, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1038, in _send_output
self.send(msg)
File "/usr/lib/python3.10/http/client.py", line 976, in send
self.connect()
File "/usr/local/lib/python3.10/dist-packages/urllib3/connection.py", line 238, in connect
self.sock = self._new_conn()
File "/usr/local/lib/python3.10/dist-packages/urllib3/connection.py", line 213, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f2d106b27a0>: Failed to establish a new connection: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/requests/adapters.py", line 486, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py", line 847, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.10/dist-packages/urllib3/util/retry.py", line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/generate (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d106b27a0>: Failed to establish a new connection: [Errno 111] Connection refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/litellm/main.py", line 1879, in completion
generator = ollama.get_ollama_response(
File "/usr/local/lib/python3.10/dist-packages/litellm/llms/ollama.py", line 194, in get_ollama_response
response = requests.post(
File "/usr/local/lib/python3.10/dist-packages/requests/api.py", line 115, in post
return request("post", url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.10/dist-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/requests/adapters.py", line 519, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=11434): Max retries exceeded with url: /api/generate (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d106b27a0>: Failed to establish a new connection: [Errno 111] Connection refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/devin/OpenDevin/agenthub/langchains_agent/utils/monologue.py", line 30, in condense
resp = llm.completion(messages=messages)
File "/usr/local/lib/python3.10/dist-packages/litellm/utils.py", line 2807, in wrapper
raise e
File "/usr/local/lib/python3.10/dist-packages/litellm/utils.py", line 2705, in wrapper
result = original_function(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/litellm/main.py", line 2094, in completion
raise exception_type(
File "/usr/local/lib/python3.10/dist-packages/litellm/utils.py", line 8296, in exception_type
raise e
File "/usr/local/lib/python3.10/dist-packages/litellm/utils.py", line 8100, in exception_type
raise ServiceUnavailableError(
File "/usr/local/lib/python3.10/dist-packages/litellm/exceptions.py", line 153, in __init__
super().__init__(
File "/usr/local/lib/python3.10/dist-packages/openai/_exceptions.py", line 81, in __init__
super().__init__(message, response.request, body=body)
AttributeError: 'NoneType' object has no attribute 'request'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/devin/OpenDevin/opendevin/controller/agent_controller.py", line 89, in step
action = self.agent.step(self.state)
File "/root/devin/OpenDevin/agenthub/langchains_agent/langchains_agent.py", line 124, in step
self._add_event(prev_action.to_dict())
File "/root/devin/OpenDevin/agenthub/langchains_agent/langchains_agent.py", line 74, in _add_event
self.monologue.condense(self.llm)
File "/root/devin/OpenDevin/agenthub/langchains_agent/utils/monologue.py", line 35, in condense
raise RuntimeError(f"Error condensing thoughts: {e}")
RuntimeError: Error condensing thoughts: 'NoneType' object has no attribute 'request'
OBSERVATION:
Error condensing thoughts: 'NoneType' object has no attribute 'request'
Exited before finishing
same behavior repeats if anything is sent to devin after killing processes.
Expected behavior Generates/responds Actual behavior Hangs Additional context config:
LLM_API_KEY="ollama"
WORKSPACE_DIR="./workspace"
LLM_BASE_URL="http://localhost:11434"
LLM_EMBEDDING_MODEL="local"
LLM_MODEL ='ollama/deepseek-coder:6.7b'
I've already cloned to the most recent version and redid setup multiple times.
I'm happy to provide any additional logs, information and context. Sorry if I forgot to mention something.
This also seems to be the same problem I was having with running opendevin in WSL but running the model server on windows: https://github.com/OpenDevin/OpenDevin/issues/435#issuecomment-2028907533
I've also had the 'NoneType' object has no attribute 'request' and MaxRetryError: HTTPConnectionPool(host='localhost', port=11434): ... errors spam using ollama before I mirrored the network in wsl.
Maybe this helps.
Mine still hangs even though I have the variable you suggest set to local, so I'm still unsure as to the reason for this behavior.
As soon as I made sure opendevin could actually reach my ollama server, my issue was solved.
If you are running both opendevin and ollama in wsl, then it's a different problem and I have no idea why you are having that issue. You are probably only getting the same errors like me because you are force closing ollama and opendevin can't reach the server anymore.
But the main issue with hanging up beforehand is probably something else, especially since you said that your GPU usage spikes.
I don't understand why it's sending 100 identical requests under enumerating steps though... (edited my last comment because I'm blind and didn't see you link your post)
Probably because it's retrying in a loop, and it always seems to stop at Step 99, even when it's working. But I still believe that the errors you see is generated only after you force-quit ollama, so I wouldn't focus too much on those errors, and try to find out what it's doing beforehand while it hangs up for a minute.
Might be an issue with ollama, as 100 is the default iterations for agent runs. I'll try to setup devin with text generation webui and openai api and see if the issue persists.
What is the best ollama model for this?, I use llama2 and OpenDevin does not do the process as he do with GPT 4, all that he shows is errors
I am not using GPT 4, and the model is irrelevant in this issue, as it is either with ollama or OpenDevin, however it is mentioned in my config. What setup do you have?