AutoGPT icon indicating copy to clipboard operation
AutoGPT copied to clipboard

Add wait_time command to allow Auto-GPT to react to rate limit errors

Open SlistInc opened this issue 1 year ago • 11 comments

Background

There are some commands that for one reason or another hit rate limits (e.g. too many API calls, search requests, twitter posts, mails being sent). This method allows Auto-GPT to react and slow down execution whenever it deems it necessary.

It would be beneficial to Auto-GPT to be able to slow down whenever it detects such a situation. See tests for a practical case in which this command helped.

Changes

Added the wait_seconds command. The command, when invoked, does nothing more than call a time.sleep. The wait was set to seconds and not minutes or milliseconds as seconds seems a timeframe that is atomic enough. Processing of API's takes ofter more than a second so millisecond would not be beneficial while minutes may be too much. My observations show me that seconds is a concept that a LLM understands quite well. Future updates may introduce wait_minutes, wait_hours or wait_days if necessary.

Documentation

I commented the code as other commands where documented. The change is minimal.

Test Plan

I tested the command in different ways: running the task with the goals of 'Wait for 5 seconds'. I also used it in my own twitter-enabled Auto-GPT and observed the command being successfuly executed after a rate limit:

`CRITICISM: We should be careful when liking tweets to avoid spamming or violating Twitter's terms of service. We should also avoid using the same hashtags repeatedly, as this can be seen as spamming.

SPEAK: We received a 'Too Many Requests' error when trying to follow a user. We need to wait for a while before trying to follow users again. ... is important to be careful when liking tweets to avoid spamming or violating Twitter's terms of service, and to avoid using the same hashtags repeatedly. ... NEXT ACTION: COMMAND = wait_seconds ARGUMENTS = {'seconds': '1800'}`

PR Quality Checklist

  • [X] My pull request is atomic and focuses on a single change.
  • [X] I have thoroughly tested my changes with multiple different prompts.
  • [X] I have considered potential risks and mitigations for my changes.
  • [X] I have documented my changes clearly and comprehensively.
  • [X] I have not snuck in any "extra" small tweaks changes

I do not have automated tests implemented but ran multiple manual tests.

SlistInc avatar Apr 17 '23 08:04 SlistInc

Please note: this is my first contribution to a public repo in general. If I did a procedural mistake I apologise and am happy to rectify it.

SlistInc avatar Apr 17 '23 08:04 SlistInc

@SlistInc CI is going to be red

nponeccop avatar Apr 17 '23 16:04 nponeccop

@SlistInc CI is going to be red

Thanks. I'm new to this, I will fix it.

SlistInc avatar Apr 17 '23 16:04 SlistInc

Should be fixed to the best of my knowledge now.

SlistInc avatar Apr 17 '23 16:04 SlistInc

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

github-actions[bot] avatar Apr 17 '23 22:04 github-actions[bot]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

github-actions[bot] avatar Apr 18 '23 05:04 github-actions[bot]

Resolved conflicts

SlistInc avatar Apr 18 '23 05:04 SlistInc

Sorry for my back and forth. It is a super simple change but I am learning about whole toolchain (e.g. linting errors, isort etc). Please advise if I am doing something wrong.

SlistInc avatar Apr 18 '23 05:04 SlistInc

Comment:

  • I am aware of the planned introduction of plugins #757 and am eagerly awaiting it.
  • I however feel that the ability to pause execution for a time period chosen by Auto-GPT is a basic functionality due to its (personally observed) welcome potential to react to rate limit errors of any kind.
  • I thus ask this command to be considered for inclusion in the basic set of functionalities. It can be further improved by more experienced members once it's in place.

SlistInc avatar Apr 18 '23 13:04 SlistInc

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

github-actions[bot] avatar Apr 19 '23 23:04 github-actions[bot]

This is a mass message from the AutoGPT core team. Our apologies for the ongoing delay in processing PRs. This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to: https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

p-i- avatar May 05 '23 00:05 p-i-

Rate limit errors are handled with exponential backoff, so we don't need it for that. But a sleep command might be useful, as long as the AI doesn't start abusing it to take breaks (which has been observed previously).

Pwuts avatar Jun 15 '23 00:06 Pwuts