OpenHands
OpenHands copied to clipboard
Use AsyncLLM for CodeActAgent and other components
This PR updates the CodeActAgent class to use the async variant of LLM completion, which should improve performance by avoiding blocking operations. The changes have been made in a way that maintains backward compatibility with existing code while enabling async operations.
Changes:
- Update CodeActAgent to use AsyncLLM and async step method
- Update base Agent class to make step method async
- Update all other agent implementations to use async step:
- VisualBrowsingAgent
- BrowsingAgent
- DummyAgent
- DelegatorAgent
- MicroAgent
- Update LLM-based condensers to use AsyncLLM and async_completion
- Update InvariantAnalyzer to use AsyncLLM and async_completion
- Update FileEditRuntimeMixin to use AsyncLLM and async_completion
- Update tests to handle async operations
The main changes were:
- Replacing LLM with AsyncLLM in imports and type hints
- Making methods async that use LLM completion
- Replacing completion() calls with async_completion()
- Updating tests to handle async operations
To run this PR locally, use the following command:
docker run -it --rm -p 3000:3000 -v /var/run/docker.sock:/var/run/docker.sock --add-host host.docker.internal:host-gateway -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:74fff3f-nikolaik --name openhands-app-74fff3f docker.all-hands.dev/all-hands-ai/openhands:74fff3f