[Resolver] API Retry on guess success
End-user friendly description of the problem this fixes or functionality that this introduces
- [ ] Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below
Give a summary of what the PR does, explaining any non-trivial design decisions
This PR partially addresses the problem of rate limit retries
-
We've replaced all
litellm.completioninstances with theLLMclass from Openhands proper. This class supports retry with the following configurable parameters:LLM_NUM_RETRIES (Default of 8)LLM_RETRY_MIN_WAIT (Default of 15 seconds)LLM_RETRY_MAX_WAIT (Default of 120 seconds)LLM_RETRY_MULTIPLIER (Default of 2)
This refactor helps centralize retry mechanism logic and applies it on
guess_successinopenhands-resolvervia theLLMclass -
Updated test cases appropriately. Also added test cases for retry on rate limit exception for
guess_success
Note: while this PR implements retry mechanism it is not a perfect solution for Anthropic's token per minute rate limit exceptions. However, if this PR is merged more sophisticated retry mechanism can be implemented openhands/llm/llm.py and openhands/llm/retry_mixin.py, which will apply to both Openhands proper and Openhands resolver.
Link of any specific issues this addresses #5030 #5087