There's a new UC Mode video showing how it works

[mdmintz]

There's a new UC Mode video showing how it works, and how it fixes bugs with undetected-chromedriver: https://www.youtube.com/watch?v=5dMFI3e85ig

Note the improvements made:

• Driver version-detection & management.
• Allowing mismatched browser/driver versions.
• Automatically changing the user agent to prevent detection. (HeadlessChrome to Chrome)
• Automatically disconnecting chromedriver from Chrome as needed. (And reconnecting)
• Multithreading tests in parallel via pytest-xdist.
• Adjusting configuration based on the environment. (Linux/Ubuntu vs Windows vs macOS)
• Explaining how to set a custom user-data-dir correctly.
• Has options for setting proxy and proxy-with-auth.

Many of these fixes can be back-ported easily to undetected-chromedriver. (Explanation in the video of how to do that, such as using requests.get(url) before navigating to a URL to know if you need to disconnect/reconnect the driver from Chrome.)

Here's a script with retries and a captcha-click failsafe for bypassing detection:

from seleniumbase import SB

with SB(uc=True) as sb:
    sb.driver.get("https://nowsecure.nl/#relax")
    sb.sleep(1)
    if not sb.is_text_visible("OH YEAH, you passed!", "h1"):
        sb.get_new_driver(undetectable=True)
        sb.driver.get("https://nowsecure.nl/#relax")
        sb.sleep(1)
    if not sb.is_text_visible("OH YEAH, you passed!", "h1"):
        if sb.is_element_visible('iframe[src*="challenge"]'):
            with sb.frame_switch('iframe[src*="challenge"]'):
                sb.click("span.mark")
                sb.sleep(2)
    sb.activate_demo_mode()
    sb.assert_text("OH YEAH, you passed!", "h1", timeout=3)

The presentation slides were autogenerated by running: https://github.com/seleniumbase/SeleniumBase/blob/master/examples/presenter/uc_presentation.py

Here are some existing examples that use --uc mode:

• https://github.com/seleniumbase/SeleniumBase/blob/master/examples/verify_undetected.py
• https://github.com/seleniumbase/SeleniumBase/blob/master/examples/uc_cdp_events.py
• https://github.com/seleniumbase/SeleniumBase/blob/master/examples/raw_uc_mode.py

[idan1109]

hi @mdmintz . I tried this code but on windows 11 using your latest version, doesnt seem to work. I never pass the check! What i'm missing? i also tried to add the headless=True flag but the browser is still opening. tried to uninstall my previous selenium version also didnt fix the problem

[mdmintz]

@idan1109 Show me the code you used. And the output of pip list. selenium is part of seleniumbase dependencies, so if you installed it correctly, it would automatically update the selenium version to the required one during install.

[wuvei]

Hi @mdmintz , I executed the code you presented above and did not pass the check. It stops after clicking the 'verify' checkbox. The version is selenium==4.14.0 seleniumbase==4.20.8. I used Chrome 119.0.6045.105 in Docker with xvfb-run.

[mdmintz]

@yczheng0 Thanks for catching that and providing details to reproduce the issue! It was a regression in version 4.20.8. Upgrade to 4.20.9 - https://github.com/seleniumbase/SeleniumBase/releases/tag/v4.20.9 for the fixed version!

[wuvei]

Thanks @mdmintz for your great work! It works.

[wuvei]

Hi @mdmintz , sorry to bother again. When the code executes to click the verify checkbox, the page will be kept in a captcha loop, i.e. always requiring clicking the checkbox. Same problem mentioned in undetected-chromedriver/#1543.

[mdmintz]

Hi @yczheng0, Assuming everything is installed correctly and you have UC Mode activated, it shouldn't be forcing you to click the checkbox at all. There are bonus methods that you can use to help:

driver.uc_open(url)
driver.uc_open_with_tab(url)
driver.uc_open_with_reconnect(url, reconnect_time=None)
driver.uc_click(selector)

(As talked about in https://www.youtube.com/watch?v=5dMFI3e85ig)

[sujaykhandekar]

@yczheng0 is correct. I also did try with all the four methods mentioned by you above. Also both the methods in youtube video including using sb context. But i think it still does get detected at "https://nowsecure.nl/" by cloudfare. It won't go past clicking on the checkbox.

[mdmintz]

@yczheng0 Which example are you running? SeleniumBase/examples/verify_undetected.py is working for me.

[wuvei]

@mdmintz I ran SeleniumBase/examples/raw_uc_mode.py, which resulted in infinite loop. SeleniumBase/examples/verify_undetected.py is working without clicking.

[mdmintz]

@yczheng0

I ran SeleniumBase/examples/raw_uc_mode.py, which resulted in infinite loop.

That shouldn't happen if you just run python raw_uc_mode.py.

SeleniumBase/examples/verify_undetected.py is working without clicking.

Great!

[diptilenka]

Hi @mdmintz i am able to get detected on a website which is cloudflare turnstile enabled i can share the link of the webpage

[mdmintz]

@diptilenka Show me the code you used. It sounds like you modified the script from examples/verify_undetected.py with a different URL, but you never modified the assertion statement to look for something else. The OH YEAH, you passed! message is unique to https://nowsecure.nl/#relax

[diptilenka]

@mdmintz Yes i changed the URL , but the cloud flare shown like this , after clicking its saying failed

[mdmintz]

@diptilenka That's a form turnstile. (I can see the password field above it.) Those work differently from turnstiles that aren't meant to be clicked at all if the system thinks you're human. The form turnstile must be clicked by both humans and bots, so it works a bit differently. See the examples for clicking form turnstiles:

• https://github.com/seleniumbase/SeleniumBase/blob/master/examples/raw_turnstile.py
• https://github.com/seleniumbase/SeleniumBase/blob/master/examples/raw_form_turnstile.py

[diptilenka]

@mdmintz I will look into those examples, but should i fill the password and username before clicking "Verify you are human" checkbox for showing success?

If we open that website in normal human way after 12 seconds it automatically shows success, and no click on that check box shown

[mdmintz]

@diptilenka

If we open that website in normal human way after 12 seconds it automatically shows success

If that's actually the case, then use sb.driver.uc_open_with_reconnect(URL, 15) so that it automatically checks the box for you. (It adds a little extra time to account for the new tab and page load.)

If that's not the case, do the turnstile click first before filling out the form. (See the examples.)

[diptilenka]

@mdmintz Thanks It worked and moving to login and other parts. will ping you once i stuck in somewhere. :)

But at first its open to verify as human and after some seconds it opens another windows and shown success.

[kaster3]

Hey, i want to tell you thanks, you're the best, did the great work. A couple of days ago i interfered with some problem, i can’t log in twitch with seleniumbase, it detects me everytime i try, whatever i did, can you give me a clue, is it possible? 😅 I use python 3.11 and macOS m series

[mdmintz]

The two main causes of getting detected in UC Mode (which are both easily handled) are:

• Timing (The UC Mode methods let you customize that when the default values aren't good enough for your env.)
• Not using the driver.uc_click(selector) method when you need to remain undetected while clicking something.

To see if UC Mode can work on your site (before adjusting for timing & special methods), load your site with the following script, which drops in a Python breakpoint() while chromedriver is disconnected from Chrome.

from seleniumbase import SB

with SB(uc=True) as sb:
    sb.driver.uc_open_with_reconnect(URL, reconnect_time="breakpoint")

From the breakpoint, manually perform that actions that you want. If you don't get detected, then good news: Once you adjust the timing and use sb.driver.uc_click(selector) when you need to click, then you can make your script work. Also note that there are other methods that you may need to use, such as sb.driver.uc_switch_to_frame("iframe") when you need to safely enter an iframe without getting detected, etc.

After you've performed your manual actions in the web browser, type c in the console and press ENTER to leave the breakpoint. (While you're in that special disconnected breakpoint, you won't be able to send webdriver commands to Chrome, but Chrome can't see webdriver either, which is how you evade detection.)

See https://github.com/seleniumbase/SeleniumBase/issues/2384#issue-2054672141 for more info about timing customizations and the special breakpoint in UC Mode.

Also see SeleniumBase/examples/raw_form_turnstile.py for an example of special clicks in UC Mode.

[kaster3]

Thanks a lot for the detailed response, but, unfortunately, it seems doesn't work even with this script and manual actions:

from seleniumbase import SB

with SB(uc=True) as sb: sb.driver.uc_open_with_reconnect(URL, reconnect_time="breakpoint")

The twitch site detected me every time and notifies me, when I click log in button or register button, after filled the forms with message: "you have a old browser version"

[mdmintz]

The UC Mode video tutorial now has over 10k views on YouTube:

---

[Tongcheng]

Thanks for the video instruction @mdmintz ! In the video you briefly mentioned we need "reconnect()" when evading CF because it has something to do with the session storage in chrome.

Could you elaborate a little more on what that means? It would be even better if there's some example for illustration purposes.

Also do you know if this is the same reason we need different user profiles when "UC" vs. "Non UC"?

[mdmintz]

@Tongcheng There's no mention of "session storage" anywhere in the video. I think you need to rewatch it.

The reconnect() is for hiding Selenium from Chrome to prevent anti-bot services from detecting Selenium.

The user_data_dir of regular Chrome vs UC Chrome are different. That's why you can't "cross the streams".

[mdmintz]

UC Mode Video 2 is ready!

https://www.youtube.com/watch?v=2pTpBtaE7SQ

[guocity]

some website detect cdc_ variable, and every-time selenium connect to chrome drive, it add the variable back, sometimes remove_cdc_props_as_needed() is not even working