OpenHands icon indicating copy to clipboard operation
OpenHands copied to clipboard

Use AsyncLLM for CodeActAgent and other components

Open tofarr opened this issue 10 months ago • 0 comments

This PR updates the CodeActAgent class to use the async variant of LLM completion, which should improve performance by avoiding blocking operations. The changes have been made in a way that maintains backward compatibility with existing code while enabling async operations.

Changes:

  • Update CodeActAgent to use AsyncLLM and async step method
  • Update base Agent class to make step method async
  • Update all other agent implementations to use async step:
    • VisualBrowsingAgent
    • BrowsingAgent
    • DummyAgent
    • DelegatorAgent
    • MicroAgent
  • Update LLM-based condensers to use AsyncLLM and async_completion
  • Update InvariantAnalyzer to use AsyncLLM and async_completion
  • Update FileEditRuntimeMixin to use AsyncLLM and async_completion
  • Update tests to handle async operations

The main changes were:

  1. Replacing LLM with AsyncLLM in imports and type hints
  2. Making methods async that use LLM completion
  3. Replacing completion() calls with async_completion()
  4. Updating tests to handle async operations

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:74fff3f-nikolaik   --name openhands-app-74fff3f   docker.all-hands.dev/all-hands-ai/openhands:74fff3f

tofarr avatar Feb 14 '25 14:02 tofarr