webarena icon indicating copy to clipboard operation
webarena copied to clipboard

Problem when redirecting to a new page in a new tab

Open taogoddd opened this issue 3 months ago • 2 comments

Dear authors,

Thanks for your brilliant work. When I tried to use this benchmark, I found two possible problems:

  1. if some action, e.g. clicking a link, will open a new page in a new tab (instead of redirecting to a page in the same tab), the "page" variable of the Env class will still be the same page (rather than the newly opened page), which means the page that is visible to the agent will still be the same page. This is a bit unreasonable because the agent will not realize the existence of the new page and just keep doing the same action on the same page.
  2. Given the first problem, there are two possible solutions: 1. assign the new page in the new tab to the "page" variable when a new tab is created, i.e. let the current page be the newly opened page; 2. tell the existence of the new page to the agent and also provide a "switch_tab" action in the action space to enable it to switch to the new tab. I think the former solution is more reasonable because it is more like the real situation. Also, "switch_tab" should also be added (it seems not in the current action space) anyway because some tasks involve multi-tab, and the agent can not do the task without the ability to switch between tabs.

Please tell me if anything is missing here. Thanks in advance!

taogoddd avatar Mar 25 '24 21:03 taogoddd