reflexion
reflexion copied to clipboard
noahshinn
actor in webshop does not intake the memory and relfexion
hi there,
I am a bit confused about the reflexion for webshop.
in code here, line 245,
https://github.com/noahshinn/reflexion/blob/main/webshop_runs/webshop_trial.py
the llm actor only intakes the base_prompt + prompt, which is the trajectory in current step, which is the left yellow block in the figure.
however, it seems that the llm actor does not intake the env_history which is the right yellow block in the figure, which contains the memory and reflexion from previous trials (if trial > 1)
may I know if I miss something ?
if this llm actor surely does not intake the memory, can it be explained that there is no gain in webshop task, as you reported in the paper ? thanks.
i forked your repo and made changes here https://github.com/noahshinn/reflexion/compare/main...yananchen1989:reflexion:yc#diff-36a02556b49e22008fa36a519bf0cde61f8343559dfde60a3c229fb72176d00fR304
not sure it should be like that. please advise.
@yananchen1989 can I ask which columns belongs to your changes? with reflex?
@yananchen1989 can I ask which columns belongs to your changes? with reflex?
hello. line 304 @theblackcat102 action = llm_chat(str(env_history) + "\n\nAction:", stop=['\n']).strip().lstrip(' ') # fix the reflexion
i guess this could be a bug in the original code which causes the wrong conclusion with regards to webshop. correct me if I miss something
my tested results. fyi
@yananchen1989 Hi, may I know which model does this result come from?
my tested results. fyi
Wow! I missed this result, but thank you for finding this issue!
