webarena

webarena copied to clipboard

Aatman21

Accessing Backing Database for Website Environments

3

Hello WebArena team, I'm interested in building on the WebArena environment and as part of this it would be helpful to be able to inspect the backing database for each...

dayvidwang

Is it possible to run any web-arena environments without using a "Pre-installed Amazon Machine Image", such as in Kubernetes? If yes, what are those environments?

ShayekhBinIslam

fuzzy match gives the wrong answer in eval

1

for task 361: our agent gave the answer: Order number 170 is Canceled, order number 189 is Pending the evaluator is using fuzzy match and evaluated our answer as wrong:...

cheng-tan

Using webarena in async environment?

3

Is there a way to use the webarena `ScriptBrowserEnv` in an async context? For some context on my use case, I'm not using the bundled `run.py`, but rather building up...

zhudotexe

Currently, trying to import `evaluation_harness` from another package causes the following error: ```pycon >>> from evaluation_harness import evaluator_router Traceback (most recent call last): [...snip] File ".../venv/lib/python3.10/site-packages/evaluation_harness/helper_functions.py", line 18, in from...

zhudotexe

It seems that we can't just copy the AMI because we don't have the right permission to the underlying snapshot. Would it be possible to make the snapshot public? Thanks...

wbeuil

Playwright Error: net::ERR_CONNECTION_TIMED_OUT, Can't connect AWS

2

Dear developer: I'm facing this error, that py playwright can't connect `EC.compute.aws`, do you have any experience with it? ``` playwright._impl._api_types.Error: net::ERR_CONNECTION_TIMED_OUT at http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:3000/ =========================== logs =========================== navigating to "http://ec2-3-131-244-37.us-east-2.compute.amazonaws.com:3000/",...

yyanhan

Hi there, Recently I met a problem with map domain. For example, if I type "Cafe near Hunt Library" in the map search domain. And enter the first choice, but...

zijianma17

The intent is: "Find the page of the longest bridge in the Western hemisphere on the map." The reference answer is Mackinac Bridge, which is incorrect. The answer (based on...

peterychang