Show rate limit issues in the UI

[rbren]

What problem or use case are you trying to solve?

I'm getting rate limited by Anthropic. But it just looks like the agent is kinda stuck while it cools down.

Describe the UX of the solution you'd like

I'd like the indicator to turn yellow, and show a relevant message about rate limits

Do you have thoughts on the technical implementation?

@tobitege has done a little preliminary work here. Basically I think we need to turn the status/badge from "agent status" to "system status"

Describe alternatives you've considered

Additional context

[tobitege]

Btw, during benches since yesterday, I received server error 502 with some html error message (lot of file edits back and forth in a short amount of time), but have a feeling that that is the error you've also experienced when getting limited?

[rbren]

Yeah exactly--I think it was due to file editing issues

[tobitege]

litellm completion calls can have a cooldown parameter with number of seconds for cooldown after hitting rate limits, i.e. it'll happen automatically without raising an exception.

[tobitege]

Just an example I found in my logs (added linebreaks for readability): (429 is default code for rate limiting in litellm)

18:04:47 - openhands:ERROR: llm.py:128 - litellm.RateLimitError: RateLimitError: OpenAIException - Error code: 429 - 
{'error': {'message': 'No deployments available for selected model, Try again in 60 seconds. Passed model=claude-3-5-sonnet@20240620. pre-call-checks=False,
allowed_model_region=n/a, cooldown_list=[(\'75365eba-c184-48b9-8195-f845d4b812ab\', 
{\'Exception Received\': \'litellm.RateLimitError: BedrockException - {"message":"Too many requests, please wait before trying again.
You have sent too many requests.  Wait before trying again."}\', \'Status Code\': \'429\'}), 
(\'0fba6cb1-2b22-45a1-9ec4-f292d74213d4\', {\'Exception Received\': \'litellm.RateLimitError: litellm.RateLimitError: VertexAIException - 
{\\n  "error": {\\n    "code": 429,\\n    "message": "Online prediction request quota exceeded for anthropic-claude-3-5-sonnet.
Please try again later with backoff.",\\n    "status": "RESOURCE_EXHAUSTED"\\n  }\\n}\\n\', \'Status Code\': \'429\'})]', 
'type': 'None', 'param': 'None', 'code': '429'}}. Attempt #1 | You can customize these settings in the configuration.

[github-actions[bot]]

OpenHands started fixing the issue! You can monitor the progress here.

[github-actions[bot]]

An attempt was made to automatically fix this issue, but it was unsuccessful. A branch named 'openhands-fix-issue-3913' has been created with the attempted changes. You can view the branch here. Manual intervention may be required.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been stalled for over 30 days with no activity.

[crishoj]

This would be very neat. Perhaps let OpenHands have another go at it?

[rkat7]

Hi, a pr's been merged to address this issue!

[neubig]

Yes, this can be closed. Thanks @rkat7 !