Introducing Looping Heuristics / Detection

[Boostrix]

Duplicates

• [X] I have searched the existing issues

Summary 💡

This is a "meta" issue to keep track of issues relating to redundant/unnecessary (infinite/endless) looping and the idea to keep track of previous arguments to detect such situations, as per: https://github.com/Significant-Gravitas/Auto-GPT/issues/3444#issuecomment-1529005391

• https://github.com/Significant-Gravitas/Auto-GPT/issues/1994
• https://github.com/Significant-Gravitas/Auto-GPT/issues/3444
• https://github.com/Significant-Gravitas/Auto-GPT/issues/3644

Idea: maintain a "call stack" that contains hashed values of each query/prompt - whenever this is used, increment a counter to detect if we're inside a loop without making much progress (the arguments will remain the same, and so will the hash) - if we are not making any progress at all, the response will also be the same (hash that too).

This should also help the agent determine if it's trying to re-solve a task that was previously tackled.

Solution: For a sub-agent, it should notify its parent agent using either the messaging API or by throwing the equivalent of an exception, so that it can be terminated/restarted: https://github.com/Significant-Gravitas/Auto-GPT/issues/1548#issuecomment-1529009589

For top-level agents, it's probably best to interrupt the loop and pursue an alternate option - which may involve human feedback: https://github.com/Significant-Gravitas/Auto-GPT/issues/3396 The cleanest method might be offering a list of options to the user (inspired by the current state of things, as per #1548), including an option for open ended feedback.

This feedback should be serialized in some form, so that the agent can easily refer back to it as per: https://github.com/Significant-Gravitas/Auto-GPT/issues/1377 The goal being to provide a means to do some form of self-assessment, as per: #305 This may involve telling the agent to log its progress to a task specific log file so that a parent agent can evaluate the log file, comparing it to the stated long-term goal.

Examples 🌈

This is just based on hashing full thoughts + new decision (command + args) and incrementing a counter every time we get to see the same "situation":

Motivation 🔦

• detect whether an agent is trying to tackle a task that it tackled previously
• detect whether it's using the same arguments and seeing the same response (=being stuck)
• get rid of unnecessary looping
• allow an agent to detect whether its work is in line with the stated goal or not
• provide a means to bail out if necessary, either informing the parent agent and/or asking for human feedback
• at the very least, use this as a means to change the problem solving strategy

[zachary-kaelan]

We could have a compact list of previously completed tasks fed into the prompt every interation. But a more robust solution would be to make it so that memory queries are done automatically every iteration and their top N results are fed into the prompt as, "You remember X, Y, Z."

[Boostrix]

I've tried to use a separate interpretation step to interpret the result of an action and modify the plan accordingly, that worked at least somewhat better than it did before. However, a number of folks now mentioned that there are 2 issues relating to pinecone memory and self feedback not working, so maybe what I am seeing is not representative currently.

[anonhostpi]

I think it would be worth creating an issue label for this as well. May prevent the need for this "meta" issue.

[anonhostpi]

There should also be a tag for JSON issues. - https://github.com/Significant-Gravitas/Auto-GPT/labels/invalid_json

Every 3rd notification I get is about issues with JSON

[Boostrix]

This is just based on hashing full thoughts + new decision (command + args) and incrementing a counter every time we get to see the same "situation":

For starters:

• should add a counter to track number of agent invocations with the same request/response resulting in the same local action
• support an outer agent specific settings to restrict this to MAX_ITERATIONS_IDENTICAL_STEPS (or via the env file)
• also, should probably consider using a configurable TIMEOUT_SECS so that the action is interrupted using a timeout error and an exact message stating it's doing some redundant.

And this stuff needs to work per agent instance, so that sub-agents can be set up accordingly.

We could have a compact list of previously completed tasks fed into the prompt every interation.

This is interesting stuff and touching on keep track of "experiences", the agent being able to remember its actions by maintaining a history of command/param tuples that worked/didn't and the associated errors/interpretation - as per: https://github.com/Significant-Gravitas/Auto-GPT/issues/3835#issuecomment-1543531885

[eyalk11]

There could be two sets.

set of already executed commands+args set of already executed commands+args that we asked the user about and he approved

If a command was already executed , then it stops and ask the user. If the user confirms, then we know that we are good to go in terms of this command and we won't stop next time. If the user rejects/ add feedback, we will stop next time.

I originally introduced the first one in https://github.com/Significant-Gravitas/Auto-GPT/pull/3914 . So I think I will delete this section and open a new PR.

https://github.com/Significant-Gravitas/Auto-GPT/pull/3914/commits/8139493dd98bab7b474cbbf89b24d6f19275d908 is the old (deleted) version.

[eyalk11]

Following the discussion with @Boostrix, I did some work on the subject. I allowed every command to have its own calculate_hash function so that it could return the hash of the file instead of the hash of the command arguments (which is the default case). https://github.com/eyalk11/Auto-GPT/commit/25d694f9b22104ae89039ab11c052332b6f3b72d .

[Boostrix]

as I mentioned on discord, I believe the first step to be coming up with ideas/challenges to trigger redundant looping and then use that as a baseline for any fixes we can come up with - no matter if it's using my original hashing based approach or something that you came up with.

Therefore, gonna ping @merwanehamadi to keep him in the loop (head of our challenges department)

[Boostrix]

FWIW, this was recently posted on discord and the article covers our looping issue: https://lorenzopieri.com/autogpt_fix/

Do they work? Nope! The problem is … AI agents do not work. The typical session of AutoGPT ends up stuck in an infinite cycle of actions, such as google something, write it to file, read the file, google again… In general, goals requiring more than 4-5 actions seem to be out of reach.

[github-actions[bot]]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[Pwuts]

Partial solution in e437065