fixes #1821 by installing required drivers and adding options to chromedriver

[psvensson]

Background

An earlier PR (#1857) was made which did fix the broken browse_website problem. Changes made to that PR led to the PR being essentially not working anymore

Changes

After some testing it is certain that the Dockerfile needs more than just installing chromium-driver. The full working new line is;

RUN apt-get -y install git libglib2.0 libnss3 libgconf-2-4 libfontconfig1 chromium-driver

In addition, to simplify merging multiple PR by the core team, the selenium settings needed for browse_website to work was removed. These are also needed.

options.add_argument( '--no-sandbox' ) options.add_argument( '--headless' )

Documentation

Test Plan

PR Quality Checklist

• [x] My pull request is atomic and focuses on a single change.
• [x] I have thoroughly tested my changes with multiple different prompts.
• [x] I have considered potential risks and mitigations for my changes.
• [x] I have documented my changes clearly and comprehensively.
• [x] I have not snuck in any "extra" small tweaks changes

[alexonab]

How about adding a HEADLESS env variable to conditionally enable or disable headless mode?

[psvensson]

1. Mention that PR number
2. Use a more descriptive title. It's ok if it's the same title as the old PR

Yes I will. I was a bit tired last evening.

[psvensson]

That is a good idea, but I feel we need to get this working asap. Please add an issue for that and tag me in it.

[Pwuts]

Closing in favor of #1473. Thanks for figuring out the packages!

[brianrhea]

I'm running into this chromedriver unexpectedly exited. Status code was: 255 error on an M1 Mac so I checked out this PR but I'm still getting the exact same error.

Happy to provide more details if it could be helpful. Thanks for working to fix this!

[Pwuts]

@brianrhea please try again with latest master. #1473 was just merged.

[brianrhea]

Bummer, same:

I should note that I am building AutoGPT with Docker and proceeding from there:

docker run -it --env-file=./.env -v $PWD/auto_gpt_workspace:/home/appuser/auto_gpt_workspace autogpt

Also, /home/appuser/.wdm/ does not exist in the container:

[Pwuts]

• Don't forget to rebuild the container
• Can you check the version of chrome and chromium_driver in the container build log?

[psvensson]

You need to do docker rm for all exited containers and docker rmi for all autgpt images before you build and test a new configuration. It takes a lot longer. but that was the only way I could be sure that my PR did actually work. But you're probably just a driver or two away to get your to work so please nab what you need :)

TO make chromium webdriver work doing web_browse inside of a Docker container you also need these libraries apt-get installed;

libgconf-2-4 libfontconfig1

And then it will work (apart from the no sandbox and headless options of course)

That's it

[psvensson]

OK, I can confirm that this issue is now fixed in master by some other PR that got merged so I'll close this. have a good day y'all

[brianrhea]

@Pwuts

Still not working for me on the lastest pull from master. I've completely wiped my containers and images and have rebuilt:

SYSTEM: Command browse_website returned: Error: Message: Service /home/appuser/.wdm/drivers/chromedriver/linux64/112.0.5615.49/chromedriver unexpectedly exited. Status code was: 255

However, my Macbook Air's Chrome version is: Version 112.0.5615.121 (Official Build) (arm64)

Perhaps some issue with mismatched architecture between my M1 and the Docker container?

[vanities]

@brianrhea, /home/appuser/.wdm/ gets created after the downloading bar gets executed when running the chrome driver.

I'm also having the same issue on master on my m1

SYSTEM:  Command browse_website returned: Error: Message: Service /home/appuser/.wdm/drivers/chromedriver/linux64/112.0.5615.49/chromedriver unexpectedly exited. Status code was: 255

I can confirm that the chromedriver is there.

[Pwuts]

Apple M1-specific issue with browse_website is tracked in #2600.

@brianrhea the version of the browser and the driver need to match quite closely, and if there is an arch mismatch I think that might be the cause of the failure.

[psvensson]

Note that the problem I was solving was a problem which was only related to running auto-gpt inside of a Docker container. So if you have problems web_browsing outside, then I have not researched that, and this was not a fix for it.

If you still have a problem with web_browsing master running auto-gpt inside of a Docker container, please do the following;

1. Remove all running or old auto-gpt containers as you can see in docker ps -a (using docker rm)
2. remove all old auto-gpt images as you can see using docker images (using docker rmi)
3. rebuild the image again and run again.

Several times when I built incrementally, I did for some reason get the latest version from docker so the above was the only was I could be sure to test every iteration.