Selenium driver for windows - Fixing session & cookie concent, and certificate parsing errors

[steinhaug]

Background

Windows, selenium driver. When using latest release I got errors when loading URLs all the time, even wrong URLs were loaded when using Chrome as selenium browser. Also there were problems with lots of pages having cookie concent popping over the page, making the logic collapse as the cookie concent page were scraped by AutoGPT instead of the real page (after the concent which wasnt done).

Changes

This change adds 2 configurable settings for the .env file, you can add a Chrome user-data-dir which is already logged in to all pages needed avoiding the Cookie concent problem. Also you can enable fixing the errors so many people are reporting on windows, and now it all works great.

Documentation

I belive the changes should be selv explanatory, I did add some comments describing aswell.

Test Plan

Should be tested on windows where you have problems with loading URL, there are loads of people reporting this. Quick google search for the typical error: https://tinyurl.com/2jffs89j

PR Quality Checklist

• [x] My pull request is atomic and focuses on a single change.
• [x] I have thoroughly tested my changes with multiple different prompts.
• [x] I have considered potential risks and mitigations for my changes.
• [x] I have documented my changes clearly and comprehensively.
• [x] I have not snuck in any "extra" small tweaks changes

I havnt added tests, as I do not know how I add tests. :(

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
docs	⬜️ Ignored (Inspect)			Apr 29, 2023 9:21am

[p-i-]

This is a mass message from the AutoGPT core team. Our apologies for the ongoing delay in processing PRs. This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to: https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

[Boostrix]

Note this looks like a useful change to be reviewed/integrated despite the ongoing re-arch effort, it's primarily config-level, with very few code changes. So definitely do consider for review/integration to help close some PRs. Thanks

[github-actions[bot]]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[netlify[bot]]

❌ Deploy Preview for auto-gpt-docs failed.

Name	Link
🔨 Latest commit	3c8b4ad91df4ee07d710e9a18681685fbfbc2e6f
🔍 Latest deploy log	https://app.netlify.com/sites/auto-gpt-docs/deploys/665f21fbdd73e0000853efdd