web-arena-x
The instruction of installation is non-reproducible
I followed the instructions strictly step by step, but the "pytest -x" didn't executed without error, which is confusing.
The following is the bug info: pytest -x ======================================================================================================================== test session starts ========================================================================================================================= platform linux -- Python 3.11.2, pytest-7.1.2, pluggy-1.3.0 rootdir: /*******/visualwebarena plugins: anyio-3.7.1, nbmake-1.4.6, asyncio-0.21.1, xdist-3.3.1 asyncio: mode=Mode.STRICT collected 65 items
tests/test_browser_env/test_action_functionalities.py .........F
============================================================================================================================== FAILURES ============================================================================================================================== ___________________________________________________________________________________________________________________________ test_key_press ___________________________________________________________________________________________________________________________
accessibility_tree_current_viewport_script_browser_env = <browser_env.envs.ScriptBrowserEnv object at 0x7fef42f17310>
def test_key_press(
accessibility_tree_current_viewport_script_browser_env: ScriptBrowserEnv,
) -> None:
env = accessibility_tree_current_viewport_script_browser_env
env.reset()
obs, success, _, _, info = env.step(
create_playwright_action(
'page.goto("https://russmaxdesign.github.io/exercise/")'
)
)
print(f"obs: {obs['text']}")
assert success
assert "textbox 'Full name'" in obs["text"]
E assert "textbox 'Full name'" in "Tab 0 (current): Exercise page for keyboard and screen reader use\n\n[4] RootWebArea 'Exercise page for keyboard and ...ox '' hasPopup: menu expanded: False\n\t\t[83] group 'Subscribe to newsletter'\n\t\t\t[240] checkbox '' checked: false"
tests/test_browser_env/test_action_functionalities.py:210: AssertionError ------------------------------------------------------------------------------------------------------------------------ Captured stdout call ------------------------------------------------------------------------------------------------------------------------ obs: Tab 0 (current): Exercise page for keyboard and screen reader use
[4] RootWebArea 'Exercise page for keyboard and screen reader use' focused: True
[87] link 'What are mammals?'
[88] link 'Classification'
[89] link 'Definitions'
[90] link 'McKenna/Bell'
[91] link 'GitHub page'
[13] main 'What are mammals?'
[15] heading 'What are mammals?'
[40] StaticText 'Mammals are any member of the group of vertebrate animals in which the young are nourished with milk from special mammary glands of the mother. In addition to milk glands, mammals are distinguished by several other unique features such as hair, the possession of a neocortex, and the presence of a chain of three tiny bones in the middle ear.'
[41] link 'Find out more about mammals'
[19] heading 'Classification'
[43] StaticText 'Mammal classification has been through several iterations since the class was first defined. No classification system is universally accepted. In 2005, Wilson and Reader provided a collection of concise, detailed information about how mammals could be classified.'
[44] link 'Find out more about classifications'
[22] heading 'Definitions'
[46] StaticText 'If Mammalia is considered as the crown group, its origin can be roughly dated as the first known appearance of animals more closely related to some extant mammals than to others. '
[48] StaticText 'Ambondro'
[49] StaticText ' is more closely related to monotremes than to therian mammals while '
[51] StaticText 'Amphilestes'
[52] StaticText ' and '
[54] StaticText 'Amphitherium'
[55] StaticText ' are more closely related to the therians; as fossils of all three genera are dated about 167 million years ago in the Middle Jurassic, this is a reasonable estimate for the appearance of the crown group.'
[14] complementary ''
[32] heading 'Mammal news signup'
[147] StaticText 'Full name'
[146] textbox '' required: False
[151] StaticText 'Email'
[150] textbox '' required: True
[156] StaticText 'Address'
[157] StaticText 'Include full street address'
[155] textbox '' required: False describedby: form-address-help
[163] StaticText 'Phone'
[164] StaticText 'Include area code'
[161] textbox '' required: False describedby: form-phone-help form-phone-error
[165] StaticText 'Error: Number must include all 8 digits'
[169] StaticText 'Favourite mammal'
[168] combobox '' hasPopup: menu expanded: False
[83] group 'Subscribe to newsletter'
[240] checkbox '' checked: false
========================================================================================================================== warnings summary ==========================================================================================================================
venv/lib/python3.11/site-packages/text_generation/types.py:45
/visualwebarena/venv/lib/python3.11/site-packages/text_generation/types.py:45: PydanticDeprecatedSince20: Pydantic V1 style @validator validators are deprecated. You should migrate to Pydantic V2 style @field_validator validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.4/migration/
@validator("best_of")
venv/lib/python3.11/site-packages/text_generation/types.py:64
/visualwebarena/venv/lib/python3.11/site-packages/text_generation/types.py:64: PydanticDeprecatedSince20: Pydantic V1 style @validator validators are deprecated. You should migrate to Pydantic V2 style @field_validator validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.4/migration/
@validator("repetition_penalty")
venv/lib/python3.11/site-packages/text_generation/types.py:70
//visualwebarena/venv/lib/python3.11/site-packages/text_generation/types.py:70: PydanticDeprecatedSince20: Pydantic V1 style @validator validators are deprecated. You should migrate to Pydantic V2 style @field_validator validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.4/migration/
@validator("seed")
venv/lib/python3.11/site-packages/text_generation/types.py:76
//visualwebarena/venv/lib/python3.11/site-packages/text_generation/types.py:76: PydanticDeprecatedSince20: Pydantic V1 style @validator validators are deprecated. You should migrate to Pydantic V2 style @field_validator validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.4/migration/
@validator("temperature")
venv/lib/python3.11/site-packages/text_generation/types.py:82
//visualwebarena/venv/lib/python3.11/site-packages/text_generation/types.py:82: PydanticDeprecatedSince20: Pydantic V1 style @validator validators are deprecated. You should migrate to Pydantic V2 style @field_validator validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.4/migration/
@validator("top_k")
venv/lib/python3.11/site-packages/text_generation/types.py:88
//visualwebarena/venv/lib/python3.11/site-packages/text_generation/types.py:88: PydanticDeprecatedSince20: Pydantic V1 style @validator validators are deprecated. You should migrate to Pydantic V2 style @field_validator validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.4/migration/
@validator("top_p")
venv/lib/python3.11/site-packages/text_generation/types.py:94
//visualwebarena/venv/lib/python3.11/site-packages/text_generation/types.py:94: PydanticDeprecatedSince20: Pydantic V1 style @validator validators are deprecated. You should migrate to Pydantic V2 style @field_validator validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.4/migration/
@validator("truncate")
venv/lib/python3.11/site-packages/text_generation/types.py:100
//visualwebarena/venv/lib/python3.11/site-packages/text_generation/types.py:100: PydanticDeprecatedSince20: Pydantic V1 style @validator validators are deprecated. You should migrate to Pydantic V2 style @field_validator validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.4/migration/
@validator("typical_p")
venv/lib/python3.11/site-packages/text_generation/types.py:106
//visualwebarena/venv/lib/python3.11/site-packages/text_generation/types.py:106: PydanticDeprecatedSince20: Pydantic V1 style @validator validators are deprecated. You should migrate to Pydantic V2 style @field_validator validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.4/migration/
@validator("top_n_tokens")
venv/lib/python3.11/site-packages/text_generation/types.py:121
//visualwebarena/venv/lib/python3.11/site-packages/text_generation/types.py:121: PydanticDeprecatedSince20: Pydantic V1 style @validator validators are deprecated. You should migrate to Pydantic V2 style @field_validator validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.4/migration/
@validator("inputs")
venv/lib/python3.11/site-packages/text_generation/types.py:127
//visualwebarena/venv/lib/python3.11/site-packages/text_generation/types.py:127: PydanticDeprecatedSince20: Pydantic V1 style @validator validators are deprecated. You should migrate to Pydantic V2 style @field_validator validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.4/migration/
@validator("stream")
venv/lib/python3.11/site-packages/pydantic/_internal/_fields.py:128 //visualwebarena/venv/lib/python3.11/site-packages/pydantic/_internal/fields.py:128: UserWarning: Field "model_id" has conflict with protected namespace "model".
You may be able to resolve this warning by setting model_config['protected_namespaces'] = ().
warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ====================================================================================================================== short test summary info ======================================================================================================================= FAILED tests/test_browser_env/test_action_functionalities.py::test_key_press - assert "textbox 'Full name'" in "Tab 0 (current): Exercise page for keyboard and screen reader use\n\n[4] RootWebArea 'Exercise page for keyboard and ...ox '' hasPopup: menu expand... !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ======================================================================================================== 1 failed, 9 passed, 12 warnings in 91.11s (0:01:31) =========================================================================================================
