OpenHands icon indicating copy to clipboard operation
OpenHands copied to clipboard

[Bug]: App breaks at Waiting for client to become ready (Session #3)

Open erikdemarco opened this issue 6 months ago • 3 comments

Is there an existing issue for the same bug? (If one exists, thumbs up or comment on the issue instead).

  • [x] I have checked the existing issues.

Describe the bug and reproduction steps

I created this issue again because this bug's issue keeps getting closed without getting fixed.

Session 1: #5968 Session 2: #7618

OpenHands Installation

Docker command in README

OpenHands Version

main

Operating System

Linux

Logs, Errors, Screenshots, and Additional Context

No response

erikdemarco avatar Jun 11 '25 04:06 erikdemarco

Hi, could you please provide the following details?

  1. Is the runtime container running? (docker ps, you should find a container named openhands-runtime-*)

  2. Enter the runtime container using: docker exec -it <runtime_container> bash Then run: curl http://127.0.0.1:<port> (Replace port with the one the OpenHands app is trying to connect to) Do you get a "Not Found", or a "Connection refused"?

  3. Run the following oneliner on both the app container and the runtime container, and share the output: docker inspect <container_name> --format='{"name":"{{ .Name }}","network_mode":"{{ .HostConfig.NetworkMode }}","networks":{{ json .NetworkSettings.Networks }}}' | jq

Thanks.

llamantino avatar Jun 11 '25 07:06 llamantino

@erikdemarco The problem with this error is that it's very generic and can happen in multiple places. Can you also provide the logs you get in the terminal please. There should be an error of some kind. Other than that follow llamantino's post will be helpful.

When you start a session, another docker container called openhands-runtime needs to start up. If that also starts up, you may need to set: --network host as well in the docker command.

That being said, since you are on Linux, have you installed Docker Desktop or just regular docker?

mamoodi avatar Jun 11 '25 13:06 mamoodi

receiving the same error on Docker with MacOS using official starting script

Terminal Log:

12:10:16 - openhands:INFO: server_config.py:53 - Using config class None INFO: Started server process [8] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:3000 (Press CTRL+C to quit) INFO: 192.168.65.1:26421 - "WebSocket /socket.io/?latest_event_id=4&conversation_id=e6e3294c989f4cf8ac507b1adb08720c&providers_set=github&session_api_key=null&EIO=4&transport=websocket" [accepted] INFO: connection open 12:10:20 - openhands:INFO: listen_socket.py:88 - sio:connect: IiVJz2Rtngjd1YZhAAAB 12:10:20 - openhands:INFO: listen_socket.py:99 - Socket request for conversation e6e3294c989f4cf8ac507b1adb08720c with connection_id IiVJz2Rtngjd1YZhAAAB 12:10:20 - openhands:INFO: listen_socket.py:124 - User None is allowed to connect to conversation e6e3294c989f4cf8ac507b1adb08720c 12:10:20 - openhands:INFO: listen_socket.py:138 - Replaying event stream for conversation e6e3294c989f4cf8ac507b1adb08720c with connection_id IiVJz2Rtngjd1YZhAAAB... 12:10:20 - openhands:INFO: listen_socket.py:166 - Finished replaying event stream for conversation e6e3294c989f4cf8ac507b1adb08720c 12:10:20 - openhands:INFO: standalone_conversation_manager.py:128 - join_conversation:e6e3294c989f4cf8ac507b1adb08720c:IiVJz2Rtngjd1YZhAAAB 12:10:20 - openhands:INFO: standalone_conversation_manager.py:258 - maybe_start_agent_loop:e6e3294c989f4cf8ac507b1adb08720c 12:10:20 - openhands:INFO: standalone_conversation_manager.py:274 - starting_agent_loop:e6e3294c989f4cf8ac507b1adb08720c 12:10:20 - openhands:INFO: listen_socket.py:183 - Successfully joined conversation e6e3294c989f4cf8ac507b1adb08720c with connection_id IiVJz2Rtngjd1YZhAAAB 12:10:20 - openhands:INFO: mcp_config.py:174 - Adding search engine to MCP config 12:10:20 - openhands:INFO: session.py:164 - Enabling pipeline condenser with: browser_output_masking(attention_window=2), llm(model="anthropic/claude-3-7-sonnet-20250219", base_url="", keep_first=4, max_size=80) 12:10:21 - openhands:INFO: docker_runtime.py:175 - [runtime e6e3294c989f4cf8ac507b1adb08720c] Waiting for client to become ready at http://host.docker.internal:30811... 12:12:22 - openhands:ERROR: agent_session.py:214 - Agent session start failed in 121.89222264289856s 12:12:22 - openhands:ERROR: session.py:221 - Error creating agent_session: [Errno 111] Connection refused Traceback (most recent call last): File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions yield File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 250, in handle_request resp = self._pool.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 256, in handle_request raise exc from None File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 236, in handle_request response = connection.handle_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 101, in handle_request raise exc File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 78, in handle_request stream = self._connect(request) ^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 124, in _connect stream = self._network_backend.connect_tcp(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/httpcore/_backends/sync.py", line 207, in connect_tcp with map_exceptions(exc_map): ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/contextlib.py", line 158, in exit self.gen.throw(value) File "/app/.venv/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions raise to_exc(exc) from exc httpcore.ConnectError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/app/openhands/server/session/session.py", line 187, in initialize_agent await self.agent_session.start( File "/app/openhands/server/session/agent_session.py", line 130, in start runtime_connected = await self._create_runtime( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/openhands/server/session/agent_session.py", line 379, in _create_runtime await self.runtime.connect() File "/app/openhands/runtime/impl/docker/docker_runtime.py", line 178, in connect await call_sync_from_async(self.wait_until_alive) File "/app/openhands/utils/async_utils.py", line 18, in call_sync_from_async result = await coro ^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 59, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/openhands/utils/async_utils.py", line 17, in coro = loop.run_in_executor(None, lambda: fn(*args, **kwargs)) ^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 338, in wrapped_f return copy(f, *args, **kw) ^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 477, in call do = self.iter(retry_state=retry_state) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 378, in iter result = action(retry_state) ^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 420, in exc_check raise retry_exc.reraise() ^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 187, in reraise raise self.last_attempt.result() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 480, in call result = fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/app/openhands/runtime/impl/docker/docker_runtime.py", line 437, in wait_until_alive self.check_if_alive() File "/app/openhands/runtime/impl/action_execution/action_execution_client.py", line 128, in check_if_alive response = self._send_action_server_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 338, in wrapped_f return copy(f, *args, **kw) ^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 477, in call do = self.iter(retry_state=retry_state) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 378, in iter result = action(retry_state) ^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 400, in self._add_action_func(lambda rs: rs.outcome.result()) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 480, in call result = fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/app/openhands/runtime/impl/action_execution/action_execution_client.py", line 125, in _send_action_server_request return send_request(self.session, method, url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 338, in wrapped_f return copy(f, *args, **kw) ^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 477, in call do = self.iter(retry_state=retry_state) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 378, in iter result = action(retry_state) ^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 400, in self._add_action_func(lambda rs: rs.outcome.result()) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 480, in call result = fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/app/openhands/runtime/utils/request.py", line 44, in send_request response = session.request(method, url, timeout=timeout, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/openhands/utils/http_session.py", line 31, in request return CLIENT.request(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 825, in request return self.send(request, auth=auth, follow_redirects=follow_redirects) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 914, in send response = self._send_handling_auth( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 942, in _send_handling_auth response = self._send_handling_redirects( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 979, in _send_handling_redirects response = self._send_single_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1014, in _send_single_request response = transport.handle_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 249, in handle_request with map_httpcore_exceptions(): ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/contextlib.py", line 158, in exit self.gen.throw(value) File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectError: [Errno 111] Connection refused

start command: docker pull docker.all-hands.dev/all-hands-ai/runtime:0.43-nikolaik

docker run -it --rm --pull=always
-e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0.43-nikolaik
-e LOG_ALL_EVENTS=true
-v /var/run/docker.sock:/var/run/docker.sock
-v ~/.openhands-state:/.openhands-state
-p 3000:3000
--add-host host.docker.internal:host-gateway
--name openhands-app
docker.all-hands.dev/all-hands-ai/openhands:0.43

dhirmadi avatar Jun 16 '25 12:06 dhirmadi

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

github-actions[bot] avatar Jul 17 '25 02:07 github-actions[bot]

This issue is stale because it has been open for 40 days with no activity. Remove the stale label or leave a comment, otherwise it will be closed in 10 days.

github-actions[bot] avatar Aug 27 '25 02:08 github-actions[bot]

This issue is stale because it has been open for 40 days with no activity. Remove the stale label or leave a comment, otherwise it will be closed in 10 days.

github-actions[bot] avatar Oct 07 '25 02:10 github-actions[bot]

This issue was automatically closed due to 50 days of inactivity. We do this to help keep the issues somewhat manageable and focus on active issues.

github-actions[bot] avatar Oct 17 '25 02:10 github-actions[bot]