OpenHands icon indicating copy to clipboard operation
OpenHands copied to clipboard

Add warm server functionality to local runtime

Open tofarr opened this issue 7 months ago • 0 comments

  • [x] This change is worth documenting at https://docs.all-hands.dev/
  • [x] Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

End-user friendly description of the problem this fixes or functionality this introduces.

This PR adds a warm server functionality to the local runtime, which significantly improves the performance of the connect method. Instead of starting a new server each time, the system can now maintain a pool of pre-warmed servers that are ready for immediate use. This reduces the startup time and provides a smoother user experience.

Two new environment variables control this functionality:

  • INITIAL_NUM_WARM_SERVERS: Controls how many warm servers will be initialized during setup
  • DESIRED_NUM_WARM_SERVERS: Controls the number of warm servers that should be maintained on an ongoing basis

Summarize what the PR does, explaining any non-trivial design decisions.

This PR implements a warm server pool for the local runtime to improve performance:

  1. Added a new global variable _WARM_SERVERS - a list of ActionExecutionServerInfo objects waiting for use
  2. Added environment variables to control the warm server functionality
  3. Modified the connect method to use warm servers when available
  4. Added methods to create warm servers in the background
  5. Modified the execute_action method to create new warm servers after an action if needed
  6. Modified the delete method to clean up warm servers when the runtime is deleted
  7. Refactored the server creation logic into a common _create_server method to reduce code duplication

The implementation ensures that:

  • When a new server is needed, it first checks if a warm server is available
  • After using a warm server, it creates a new one in the background if needed
  • All resources are properly cleaned up when the runtime is deleted

Link of any specific issues this addresses:

N/A


To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:16d6f35-nikolaik   --name openhands-app-16d6f35   docker.all-hands.dev/all-hands-ai/openhands:16d6f35

tofarr avatar Jun 10 '25 03:06 tofarr