SeleniumBase icon indicating copy to clipboard operation
SeleniumBase copied to clipboard

Documenting the `selenium-stealth` integration with UC Mode

Open mdmintz opened this issue 1 year ago • 14 comments

Documenting the selenium-stealth integration with UC Mode

(Info: selenium-stealth lets you mask your web browser's fingerprint and WebGL information.)

Prerequisites: selenium-stealth must be installed separately: pip install selenium-stealth

Here's an example of the selenium-stealth integration with the Driver() manager and UC Mode:

from seleniumbase import Driver
from selenium_stealth import stealth

driver = Driver(uc=True)
stealth(
    driver,
    languages=["en-US", "en"],
    vendor="Google Inc.",
    platform="Win32",
    webgl_vendor="Intel Inc.",
    renderer="Intel Iris OpenGL Engine",
    fix_hairline=True,
)
driver.get("https://browserleaks.com/webrtc")
driver.sleep(10)
driver.quit()

selenium-stealth also integrates with SeleniumBase BaseCase formats:

Use Syntax Format 2 to override the setUp() method so that selenium-stealth settings are used:

from seleniumbase import BaseCase
from selenium_stealth import stealth

class BaseTestCase(BaseCase):
    def setUp(self):
        super().setUp()
        stealth(self.driver,
            languages=["en-US", "en"],
            vendor="Google Inc.",
            platform="Win32",
            webgl_vendor="Intel Inc.",
            renderer="Intel Iris OpenGL Engine",
            fix_hairline=True,
        )

Then have your test classes inherit BaseTestCase instead of BaseCase. (See SeleniumBase/help_docs/syntax_formats.md#sb_sf_02 for details.)

To activate UC Mode in BaseCase formats, use --uc as a pytest command-line option:

pytest --uc

For general information about selenium-stealth, see:

  • https://github.com/diprajpatra/selenium-stealth
  • https://stackoverflow.com/q/70265306/7058266

For general information about UC Mode, see https://github.com/seleniumbase/SeleniumBase/issues/2213

mdmintz avatar Dec 31 '23 22:12 mdmintz

Stealth hasn't been updated in a while, is it still viable?

AlexPaiva avatar Jan 03 '24 00:01 AlexPaiva

@AlexPaiva Some people are still using it, so it's probably still valid, even though it hasn't been updated in awhile.

But at this point, selenium-stealth is mainly just for fingerprint-masking. The part with avoiding selenium-detection is fully being done by seleniumbase UC Mode.

mdmintz avatar Jan 03 '24 01:01 mdmintz

@ AlexPaiva有些人仍然在使用它,所以它可能仍然有效,即使它有一段时间没有更新。

但在这一点上,selenium-stealth主要是为了指纹掩蔽。 避免硒检测的部分完全由seleniumbaseUC模式完成。

I think in this case, although the fingerprint has been changed, the fingerprint is fixed every time. If only it were random fingerprints every time

xipeng5 avatar May 20 '24 09:05 xipeng5

Thank you very much for your reply. Your library has benefited me greatly. Sincerely thank you, but I have encountered some issues. I tried to register an account using your library, but every time it failed. I used a proxy IP and set the browser's fingerprint and other parameters, but none of them worked. I don't know where the problem is. If it doesn't take up your time, I hope you can help me take a look at the code. Below is the code snippet link. Thank you very much and I wish you a happy life。https://github.com/xipeng5/testap/blob/main/test.py

xipeng5 avatar May 21 '24 07:05 xipeng5

how use stealth with SB(from seleniumbase import SB) and uc=True? adding

stealth(sb.driver,
        languages=["en-US", "en"],
        vendor="Google Inc.",
        platform="Win32",
        webgl_vendor="Intel Inc.",
        renderer="Intel Iris OpenGL Engine",
        fix_hairline=True,
       )

don't works

kitsune0n avatar Jun 25 '24 21:06 kitsune0n

@kitsune0n If the driver needs to disconnect (eg, a uc_open method is called) or if going to a site with a Cloudflare CAPTCHA (eg. via driver.get(), where the disconnect is automatic) then selenium-stealth settings are reset. Unless the maintainer of https://github.com/diprajpatra/selenium-stealth makes changes, there's nothing I can do from my end to help.

Also, selenium-stealth is not needed to bypass CAPTCHAs: Regular seleniumbase UC Mode can do that alone.

mdmintz avatar Jun 25 '24 21:06 mdmintz

I have used selenium-stealth, but in UC mode, there is always a need for connect and reconnect methods. Do I need to apply stealth each time? because as you said it resets if I use uc_open or uc_click or any disconnect method use. Is there any way to always attach again and all the info with any customize?

 with SB(uc=True) as sb:
        stealth(
            sb.driver,
            user_agent=device.get('user_agent'),
            languages=["en-US", "en"],
            vendor=device.get('vendor'),
            platform=device.get('platform'),
            webgl_vendor=device.get('webgl_vendor'),
            renderer=device.get('renderer'),
            fix_hairline=device.get('fix_hairline'),
        )
        sb.set_window_size(width=device.get('width'), height=device.get('height'))
        sb.uc_open('https://amiunique.org/')
        time.sleep(50)

In SB there is agent set method but not for timezone, we can set the timezone external raw script but it reset while disconnecting the driver, please let me know if there is a change to customize SB to override the method of reconnect, as I have used SB context-manager base so I need to customize uc_open, uc_click and reconnect method so I can execute timezone method each time

sb.execute_cdp_cmd('Emulation.setTimezoneOverride', tz_params)

PankajSavaliya avatar Jul 05 '24 09:07 PankajSavaliya

https://github.com/seleniumbase/SeleniumBase/discussions/2856#discussioncomment-9789603:

When UC Mode disconnects the driver from Chrome to evade detection, it also resets any settings made by selenium-stealth.

Also, it appears that using selenium-stealth to modify settings will get detected as a possible attempt to mask your fingerprint. (There is little that I can do there, as that is an external library.)

Your best bet to avoid detection is by just using regular SeleniumBase UC Mode:

from seleniumbase import SB

with SB(uc=True, incognito=True) as sb:
    sb.uc_open_with_reconnect("https://pixelscan.net/", 10)
    breakpoint()

If you want timezone changes to be permanent, use a Chrome extension that changes your timezone.

mdmintz avatar Jul 05 '24 12:07 mdmintz

#2856 (comment):

When UC Mode disconnects the driver from Chrome to evade detection, it also resets any settings made by selenium-stealth.

Also, it appears that using selenium-stealth to modify settings will get detected as a possible attempt to mask your fingerprint. (There is little that I can do there, as that is an external library.)

Your best bet to avoid detection is by just using regular SeleniumBase UC Mode:

from seleniumbase import SB

with SB(uc=True, incognito=True) as sb:
    sb.uc_open_with_reconnect("https://pixelscan.net/", 10)
    breakpoint()

If you want timezone changes to be permanent, use a Chrome extension that changes your timezone.

Yes, I want to change the timezone permanently, I have already removed that library because it's not updated.

Can you please how to do with extension with SB? just give me a hint as we can install dynamically in driver but how? @mdmintz

PankajSavaliya avatar Jul 05 '24 13:07 PankajSavaliya

@PankajSavaliya For changing the timezone, you'll have to find an extension that does it. I hear they exist, but I don't know one off the top of my head directly.

mdmintz avatar Jul 05 '24 13:07 mdmintz

How to add the extension in SB if I have an extension so how to give the path of extension to SB? @mdmintz

PankajSavaliya avatar Jul 05 '24 13:07 PankajSavaliya

Adding an extension: https://github.com/seleniumbase/SeleniumBase/blob/1c526f1c02a9c7f4ec538d1600829acff44aa05e/seleniumbase/plugins/sb_manager.py#L74

mdmintz avatar Jul 05 '24 13:07 mdmintz

@mdmintz I have tried a lot can you please help me find this kind of custom extension? I have been trying for 6 hours but no luck finding for set to the timezone and datetime via extension.

PankajSavaliya avatar Jul 05 '24 19:07 PankajSavaliya

@mdmintz

I have created an extension and set the path to extension_dir, and it is working perfectly. However, there is one issue: in my extension, I'm using the below command in the script, and due to this debugger command, this window continues to show up. Is there a way to hide it and make it undetectable?

chrome.debugger.sendCommand({ tabId: tabId }, 'Emulation.setTimezoneOverride'

debugging

PankajSavaliya avatar Jul 06 '24 13:07 PankajSavaliya