undetected-chromedriver
undetected-chromedriver copied to clipboard
Authenticated proxy with UC (without selenium-wire)
I have tried to implement HTTP proxy authentication in chrome webdriver using the extension method. It works well with plain webdriver, and selenium-stealth as well but not with UC.
How do I use authenticated proxy servers with UC?
Note: I cannot use selenium-wire
, cloudflare had detected TLS fingerprints of mitmproxy
on my scraping IPs/sites and immediately throws the access-denied page when I use selenium-wire.
UC v3.4 used to have a proxy parameter, but it has been removed unfortunately in later versions.
Try selenium-base?
I have successfully used mitmproxy to access https://nowsecure.nl/ without being detected. Everything worked normally, and there were no issues with the TLS fingerprints or any other security measures. So, it seems like mitmproxy can be a viable solution for your use case.
My issues arose from a particular site blacklisting my scraper based on selenium-wire
. But the IP addresses were and not banned (I can still access the site using those proxy IPs with a real browser or selenium+UC). With selenium-wire, CF immediately throws the "access denied" page. Additionally, I'm using a few commercial DC proxy providers which enforce user-login authentication.
(I can now confirm this is a site-specific issue. selenium-wire
works fine on other websites.)
Anyhow, I have switched to selenium-stealth
(seems to be abandoned) which works with proxy authentication (using the custom chrome extension approach). This method doesn't work with UC, and I had spent half a day trying to get it to function with UC.
use chrome_options.add_argument(r'--load-extension=absolute_path_to_unzip_exctension_folder')
chrome_options.add_extension(pluginfile)
not work
I recommend. botanasaurs .