webarena
webarena copied to clipboard
Problem when redirecting to a new page in a new tab
Dear authors,
Thanks for your brilliant work. When I tried to use this benchmark, I found two possible problems:
- if some action, e.g. clicking a link, will open a new page in a new tab (instead of redirecting to a page in the same tab), the "page" variable of the Env class will still be the same page (rather than the newly opened page), which means the page that is visible to the agent will still be the same page. This is a bit unreasonable because the agent will not realize the existence of the new page and just keep doing the same action on the same page.
- Given the first problem, there are two possible solutions: 1. assign the new page in the new tab to the "page" variable when a new tab is created, i.e. let the current page be the newly opened page; 2. tell the existence of the new page to the agent and also provide a "switch_tab" action in the action space to enable it to switch to the new tab. I think the former solution is more reasonable because it is more like the real situation. Also, "switch_tab" should also be added (it seems not in the current action space) anyway because some tasks involve multi-tab, and the agent can not do the task without the ability to switch between tabs.
Please tell me if anything is missing here. Thanks in advance!
One example of opening a new page in a new tab is when the agent tries to click the "Go to Advanced Reporting" button in the shopping admin website like this:
Thank you very much, we are fixing this issue #102