webarena
webarena copied to clipboard
Wrong scroll action specification in the prompts vs. in the parsing function
Hi, I'm reporting just a small bug, the prompts all say to use "scroll [direction=up|down]" but the parsing function expects "scroll [up|down]". This way the parsing always fails for the scroll action because the model always generates "scroll [direction=up|down]".
@shuyanzhou
On the other hand, judging from you GPT4 experiment logs, this was not a problem for GPT4. Maybe this is just specific for the Gemini I was using. It always tried to generate the action in the wrong format, e.g. scroll [direction=down]. In that case this issue might be irrelevant