BrowserGym

BrowserGym copied to clipboard

Browsergym hangs for hours when trying to save summary info for last run

3

When running on the test split (381/381, i think task id 811 but not sure), it hangs at the following stages: > 2024-12-17 02:32:41,293 - 345050 - browsergym.experiments.loop - INFO...

xhluca

jardinetsouffleton

GAIA

2

- add gaia and gaia eval (based on assistantbench pr - https://github.com/ServiceNow/BrowserGym/pull/186/) - refactor writing predictions to jsonl to a utils file - fix assistantbench readme

oriyor

## Pull Request: Integrate WebCanvas Key Node Evaluation and Mind2web-live Benchmark into BrowserGym ### Description This PR officially integrates the **WebCanvas key node evaluation** and the **Mind2web-live benchmark** into **BrowserGym**....

han032206

Moving version fetching to BrowserGym. It feels too hardcoded atm, maybe we could deduce the pkg name automatically ? Maybe some kind of regex on the pkg name

TLSDC

Add mouse position into env observation

4

Hi, thanks for the project! I'm trying to implement and experiment with coordinate-based actions from `browsergym` and it would be useful if the environment exposes this info via the observation....

ryanhoangt

jardinetsouffleton

WebArena Shopping evaluator issues

2

There is an issue with some WebArena shopping tasks: - On task 275: it's a search task where the agent is asked to search for "xbox". So the reference URL...

imenelydiaker

WebArena Verified

2

Introduce the webarena_verified benchmark. - tasks are registered with this template: `webarena_verified.{intent_template_id}.{task_id}` - new `WebArenaVerifiedTask` class overrides the `setup()` function of `GenericWebArenaTask` to: - use the webarena_verified evaluator - load...

NicolasAG

Hi, In the following step, the action `mouse_click(1219, 228, button='left')` is not executed properly on WorkArena L1, even if the blue cursor shows the mouse is on the right element....

imenelydiaker