agent-zero icon indicating copy to clipboard operation
agent-zero copied to clipboard

Browser-agent not working - Playwright error

Open arsaboo opened this issue 9 months ago • 27 comments

Tried using browser-agent and I am getting this error:

Agent 0: Generating
{
    "thoughts": [
        "I need to scrape the top 5 news items from Hacker News.",
        "I will use a browser agent to navigate and extract the information."
    ],
    "tool_name": "browser_agent",
    "tool_args": {
        "message": "Open https://news.ycombinator.com/ and extract the titles and links of the top 5 news items. End task.",
        "reset": "true"
    }
}
Agent 0: Using tool 'browser_agent'
Message: Open https://news.ycombinator.com/ and extract the titles and links of the top 5 news items. End task.
Reset: true
ERROR    [browser] Failed to initialize Playwright browser: BrowserType.launch: Executable doesn't exist at /root/.cache/ms-playwright/chromium_headless_shell-1148/chrome-linux/headless_shell
╔════════════════════════════════════════════════════════════╗
║ Looks like Playwright was just installed or updated.       ║
║ Please run the following command to download new browsers: ║
║                                                            ║
║     playwright install                                     ║
║                                                            ║
║ <3 Playwright Team                                         ║
╚════════════════════════════════════════════════════════════╝
Traceback (most recent call last):
Traceback (most recent call last):
  File "/a0/agent.py", line 310, in monologue
    tools_result = await self.process_tools(agent_response)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/a0/agent.py", line 637, in process_tools
    response = await tool.execute(**tool_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/a0/python/tools/browser_agent.py", line 183, in execute
    result = await task.result()
             ^^^^^^^^^^^^^^^^^^^
  File "/a0/python/helpers/defer.py", line 114, in result
    return await loop.run_in_executor(None, _get_result)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/a0/python/helpers/defer.py", line 106, in _get_result
    result = self._future.result(timeout)  # type: ignore
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/a0/python/helpers/defer.py", line 83, in _run
    return await self.func(*self.args, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>  9 stack lines skipped <<<
  File "/opt/venv/lib/python3.11/site-packages/browser_use/browser/browser.py", line 89, in _init
    browser = await self._setup_browser(playwright)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/browser_use/browser/browser.py", line 152, in _setup_browser
    browser = await playwright.chromium.launch(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/playwright/async_api/_generated.py", line 14404, in launch
    await self._impl_obj.launch(
  File "/opt/venv/lib/python3.11/site-packages/playwright/_impl/_browser_type.py", line 95, in launch
    Browser, from_channel(await self._channel.send("launch", params))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 528, in wrap_api_call
    raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
playwright._impl._errors.Error: BrowserType.launch: Executable doesn't exist at /root/.cache/ms-playwright/chromium_headless_shell-1148/chrome-linux/headless_shell
╔════════════════════════════════════════════════════════════╗
║ Looks like Playwright was just installed or updated.       ║
║ Please run the following command to download new browsers: ║
║                                                            ║
║     playwright install                                     ║
║                                                            ║
║ <3 Playwright Team                                         ║
╚════════════════════════════════════════════════════════════╝

I am running agent-zero using docker with Ollama.

arsaboo avatar Feb 08 '25 14:02 arsaboo

after you ask the agent to install playwright, then it still does not work as instead of opening the urls new card 'about:blank' is being opened and it throws out an error @frdel

Jukeman9 avatar Feb 08 '25 17:02 Jukeman9

@Jukeman9 you gave me the idea and I asked the agent to install playwright and the dependencies and now browser-agent is working. However, it is not giving the output:

Image

Text	
'NoneType' object has no attribute 'strip'


Traceback (most recent call last):
Traceback (most recent call last):
  File "/[a0](http://192.168.2.162:50001/#)/[agent.py](http://192.168.2.162:50001/#)", line 310, in monologue
    tools_result = await self.process_tools(agent_response)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/[a0](http://192.168.2.162:50001/#)/[agent.py](http://192.168.2.162:50001/#)", line 639, in process_tools
    await tool.after_execution(response)
  File "/[a0](http://192.168.2.162:50001/#)/[python](http://192.168.2.162:50001/#)/[helpers](http://192.168.2.162:50001/#)/[tool.py](http://192.168.2.162:50001/#)", line 34, in after_execution
    text = response.message.strip()
           ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'strip'


AttributeError: 'NoneType' object has no attribute 'strip'

arsaboo avatar Feb 08 '25 18:02 arsaboo

For some reason the script expects chromium_headless_shell-1148 but chromium_headless_shell-1155 is available instead. Instead of telling the AI to install playwright I did it myself manually (/a0/docker/run/fs/ins/install_playwright.sh or /ins/install_playwright.sh, can't remember) and it replaced the files with expected ones. Anyway, the same result as you, sorta works but not really.

romano avatar Feb 08 '25 21:02 romano

@Jukeman9 you gave me the idea and I asked the agent to install playwright and the dependencies and now browser-agent is working. However, it is not giving the output:

Image

Text 'NoneType' object has no attribute 'strip'

Traceback (most recent call last): Traceback (most recent call last): File "/a0/agent.py", line 310, in monologue tools_result = await self.process_tools(agent_response) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/a0/agent.py", line 639, in process_tools await tool.after_execution(response) File "/a0/python/helpers/tool.py", line 34, in after_execution text = response.message.strip() ^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'strip'

AttributeError: 'NoneType' object has no attribute 'strip'

Exact same error, got stuck at the exact same point. I actually tried to run the prompt in the video, so "go to agent zero github and tell me what's the latest release", and I noticed in the reasoning that it was generating the wrong url (something like github.com/AgentZero). I tried inserting the right url directly in the prompt but got to the same error.

OnirM avatar Feb 18 '25 10:02 OnirM

Same error

Image

Than I just asked to install playwright (but, hey, you're actually an agent and could have done it yourself lol)

Image

He back to main task, my prompt and than new error

Image

Than I restarted agent (big button Restart) and asked to back to main task and got the same "Looks like Playwright was just installed or updated." error

What am I doing wrong? =/

stepahin avatar Feb 19 '25 14:02 stepahin

Hi everyone @romano @arsaboo @OnirM @stepahin @Jukeman9

Just for now and temporarily, the browser-agent is gonna work if you pull the frdel/agent-zero-run:testing Docker image. We're investigating the issue at hand but the testing image is gonna work, at least with known models (Llama, gpt-4o, gemini-2.0-flash).

3clyp50 avatar Feb 19 '25 16:02 3clyp50

UPDATE: The frdel/agent-zero-run:latest has been updated with the fix.

3clyp50 avatar Feb 19 '25 23:02 3clyp50

Not quite:

Image

arsaboo avatar Feb 25 '25 20:02 arsaboo

Not quite:

Image

same error here.

OnirM avatar Feb 28 '25 11:02 OnirM

Make sure you have the updated version of the docker image. This is fixed already.

frdel avatar Feb 28 '25 11:02 frdel

just re-downloaded the the latest docker image, same error:

Image

OnirM avatar Feb 28 '25 11:02 OnirM

Please:

  1. ask the agent what version of playwright is installed
  2. Copy the A0 version from bottom left corner
  3. Paste both here

Thank you.

frdel avatar Feb 28 '25 11:02 frdel

A0 version: Version M v0.8.1.2-7 25-02-19 21:34

ok when asked about the version of playwright, in the reasoning it said it was not installed, then proceeded to install it:

Image

now I'm on version 1.50.1

tried to ask again to browse the repo but got again the same error

OnirM avatar Feb 28 '25 12:02 OnirM

It's not NPM package, it's a python package, it should look like this:

Image

frdel avatar Feb 28 '25 12:02 frdel

Image

arsaboo avatar Feb 28 '25 19:02 arsaboo

I am facing the same issue as well. Attached is the image showing the A0 version and Playwright version.

Image

alloke613 avatar Mar 08 '25 20:03 alloke613

Same issue. This is critical for any usability of agent. Image Image

npnpatidar avatar Mar 09 '25 03:03 npnpatidar

I just downloaded the latest image, told it to go to weather.com and it works. No additional setup, no folder mapping.

Make sure you have the latest image and that you're not reusing folders from previous versions.

My version is: Version M v0.8.1.2-7 25-02-19 21:34 Playwright: 1.49.0

Image

frdel avatar Mar 11 '25 18:03 frdel

Thank you. It is working.

The issue was with web browser model. It seems all models do not work. Gemini , Mistral etc which are freely available do not work. I tried lots of them and it seems some from Groq are working with vision off but the rate limit on them is very tight so not very useful. It would be great if gemini and mistral can be made working.

I would like to propose that although OpenAI api is standard, it would be great if gemini or mistral can be set as default.

  1. Both provide embedding models as well as vision models
  2. Both have free api available which can be great for new users who do not want to pay just to test.
  3. Everybody has google account so they can get api key easily and this would make agent-zero plug and play.

npnpatidar avatar Mar 12 '25 05:03 npnpatidar

I found a solution to this issue that works for my particular use case (I'm using Google and "gemini-2.0-flash" as my browser_model):

TL;DR: The class that has to be used for the instantiation of the Gemini browser model is ChatGoogleGenerativeAI instead of GoogleGenerativeAI.

Explanation: this is in fact a problem with Browser Use and its supported models Groq is not supported yet so it won't work here in Agent Zero either ...

In the example for Gemini they use ChatGoogleGenerativeAI

The code in Agent Zero uses GoogleGenerativeAI

The change is not as simple as replacing GoogleGenerativeAI with ChatGoogleGenerativeAI in that line because that would affect all the Gemini models, not just the browser model, and I'm not sure how different the two classes are.

To verify that what I say is true this toy example can be run in the conda a0 environment:

from langchain_google_genai import GoogleGenerativeAI
from browser_use import Agent
from pydantic import SecretStr
import os
from dotenv import load_dotenv
load_dotenv()

api_key = os.getenv("GEMINI_API_KEY")

# Initialize the model
llm = GoogleGenerativeAI(model='gemini-2.0-flash', api_key=SecretStr(os.getenv('GEMINI_API_KEY')))

import asyncio

async def main():
    # Create agent with the model
    agent = Agent(
        task="go to https://www.google.com",
        llm=llm,
    )
    result = await agent.run()
    print(result)

asyncio.run(main())

This won't work, but if you substitute GoogleGenerativeAI with ChatGoogleGenerativeAI, it will.

What I've done locally is modify browser_agent.py to do something similar to this:

if self.agent.config.browser_model.provider == models.ModelProvider.GOOGLE:
    model = ChatGoogleGenerativeAI(model=model_name, api_key=SecretStr(api_key),
                                      safety_settings={HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE}, **kwargs)

and it works ...

If you want, I can create a PR with this change, but I'm not sure this is the best way to fix it because, as I mentioned, in the end, it's a Browser Use issue.

Regards

pvm30 avatar Mar 12 '25 18:03 pvm30

Instead of updating browser_agent.py, change was required in models.py. I have created a pull request to enable use of gemini in both embedding as well as browser model.

npnpatidar avatar Mar 13 '25 08:03 npnpatidar

Instead of updating browser_agent.py, change was required in models.py. I have created a pull request to enable use of gemini in both embedding as well as browser model.

Even with this pull I get errors like "GenerateContentRequest.tools[0].function_declarations[0].parameters.properties[action].items.properties[done].properties: should be non-empty for OBJECT type" when using google gemini-2.0-flash. Before that playwright was stuck at about:blank pages.

Spadav avatar Mar 13 '25 20:03 Spadav

Instead of updating browser_agent.py, change was required in models.py. I have created a pull request to enable use of gemini in both embedding as well as browser model.

I have to admit I was wrong and a little bit too cautious...

I haven't built the Docker image, but I can confirm that just by applying this simpler change, everything seems to keep working. Nothing appears to break by returning a ChatGoogleGenerativeAI instead of GoogleGenerativeAI from get_chat_model and get_utility_model.

Regards

Image

pvm30 avatar Mar 14 '25 18:03 pvm30

I am new to agent-zero I installed docker base setup with latest after I configure google the command task are working. Browser base task not working. I really have no clue what Web Browser model name to give for google.

sjkeerthi avatar Mar 23 '25 19:03 sjkeerthi

I got it to work when I started using the new mistal model locally with lm studio, I don't know if the issue was with ollama or with the qwen model I was using.

Spadav avatar Mar 23 '25 19:03 Spadav

still not working, playwright is 1.49.0. Use openrouter gpt-4o. A0 version 0.8.2.1 25-04-07

Update: Took care of it, after I told the agent to pip install playwright --upgrade

NeverOccurs avatar Apr 23 '25 15:04 NeverOccurs

I was facing the same problem with the browser_agent in the current version 0.8.3.1, when I asked Agent Zero to open a web page URL in the web browser. At first, it complained about playwright being missing, I asked to fix the error and it installed playwright and other dependencies, then I asked again to open the web page and I got the NoneType error below:

Image

So, I asked Agent Zero to fix the error and apparently it did it.

Image

Image

It changed the code inside /a0/python/helpers/tools.py from this

Image

to this

Image

I don't know if the "solution" is correct or not, but I thought it was worth reporting it here.

Harvester62 avatar May 10 '25 23:05 Harvester62