OpenHands icon indicating copy to clipboard operation
OpenHands copied to clipboard

Detect and stop infinite loops

Open R3verseIN opened this issue 1 year ago • 7 comments

Here is the output

==============
STEP 0

        ACTION:
        AgentThinkAction(thought='I think I will start by running `ls` to see what files are in the current directory.', runnable=False)


==============
STEP 1

        ACTION:
        AgentThinkAction(thought='I should start by running `ls` to see what files are in the current directory. Then, I can think about how to organize them and make progress on my task.', runnable=False)


==============
STEP 2

        ACTION:
        AgentThinkAction(thought='Okay, so I need to figure out how to write the code for my task. I think I will start by reading the documentation for the project and familiarizing myself with the tools and libraries available.', runnable=False)


==============
STEP 3

        ACTION:
        AgentThinkAction(thought='I should start by familiarizing myself with the project documentation and tools available.', runnable=False)


==============
STEP 4

        ACTION:
        AgentThinkAction(thought='Okay, so I need to figure out how to write the code for my task. I think I will start by reading the documentation for the project and familiarizing myself with the tools and libraries available.', runnable=False)


==============
STEP 5

        ACTION:
        AgentThinkAction(thought='I should start by familiarizing myself with the project documentation and tools available.', runnable=False)


==============
STEP 6

        ACTION:
        AgentThinkAction(thought='Okay, so I need to figure out how to write the code for my task. I think I will start by reading the documentation for the project and familiarizing myself with the tools and libraries available.', runnable=False)


==============
STEP 7

        ACTION:
        AgentThinkAction(thought='I should start by familiarizing myself with the project documentation and tools available.', runnable=False)


==============
STEP 8

        ACTION:
        AgentThinkAction(thought='I think I should start by familiarizing myself with the project documentation and tools available.', runnable=False)


==============
STEP 9

        ACTION:
        AgentThinkAction(thought='I should start by familiarizing myself with the project documentation and tools available.', runnable=False)


==============
STEP 10

        ACTION:
        AgentThinkAction(thought='I should start by familiarizing myself with the project documentation and tools available.', runnable=False)


==============
STEP 11

        ACTION:
        AgentThinkAction(thought='I should start by familiarizing myself with the project documentation and tools available.', runnable=False)


==============
STEP 12

        ACTION:
        AgentThinkAction(thought='I should start by familiarizing myself with the project documentation and tools available.', runnable=False)


==============
STEP 13

        ACTION:
        AgentThinkAction(thought='Okay, so I need to figure out how to write the code for my task. I think I will start by familiarizing myself with the project documentation and tools available.', runnable=False)


==============
STEP 14

        ACTION:
        AgentThinkAction(thought='I should start by familiarizing myself with the project documentation and tools available.', runnable=False)


==============
STEP 15
^C^C^C^C
        ACTION:
        AgentThinkAction(thought='I think I should start by familiarizing myself with the project documentation and tools available.', runnable=False)

Here is the runned command

export LLM_EMBEDDING_MODEL="llama2"
export LLM_BASE_URL="http://localhost:11434"
export LLM_MODEL="ollama/llama2"
export OPENAI_API_KEY=""
export WORKSPACE_DIR="/home/r3versein/work/"
uvicorn opendevin.server.listen:app --port 3000

R3verseIN avatar Mar 29 '24 01:03 R3verseIN

Someone may wanna check out issue #117. A dual-model approach may solve the issue by micromanaging.

rezzie-rich avatar Mar 29 '24 05:03 rezzie-rich

yeah, i have the same issue

Likhithsai2580 avatar Mar 29 '24 09:03 Likhithsai2580

We've mostly been testing on OpenAI--llama (especially local 7B llama) is inevitably going to do worse :/

Might be worth creating a special agent for the less-capable LLMs

rbren avatar Mar 29 '24 14:03 rbren

We've mostly been testing on OpenAI--llama (especially local 7B llama) is inevitably going to do worse :/

Might be worth creating a special agent for the less-capable LLMs

Then what to do now?

R3verseIN avatar Mar 29 '24 16:03 R3verseIN

I'd suggest trying one of the 70B models, or using a paid LLM like GPT or Claude.

You're also welcome to take a crack at building a new agent!

rbren avatar Mar 29 '24 19:03 rbren

Changing the purpose of this issue to detecting and killing infinite loops

If the controller notices the exact same action 3+ times, we should kill the loop

rbren avatar Mar 29 '24 23:03 rbren

Might be worth creating a special agent for the less-capable LLMs

Special agent sounds great. Or maybe special prompts. Related issue: https://github.com/OpenDevin/OpenDevin/issues/1134

li-boxuan avatar Apr 16 '24 02:04 li-boxuan