undetected-chromedriver icon indicating copy to clipboard operation
undetected-chromedriver copied to clipboard

Can't run multiple instances

Open Mikayel0 opened this issue 2 years ago • 38 comments

Can not run more than one browser

import undetected_chromedriver as uc
import os
from concurrent import futures

def browser(thread):
    uc.Chrome()
    print(thread)
    input()

def main():
    os.system('cls')
    threads = input("How much threads you want: ")
    with futures.ThreadPoolExecutor() as executor: 
        future_test_results = [ executor.submit(browser, i)  
        for i in range(int(threads)) ]
        for future_test_result in future_test_results: 
            try:        
                test_result = future_test_result.result()          
            except Exception as exc:
                print('thread generated an exception: {:0}'.format(exc))

if __name__ == "__main__":
    main()

When second browser is opening

FileExistsError: [WinError 183] Cannot create a file when that file already exists:
'C:\\Users\\User\\appdata\\roaming\\undetected_chromedriver\\undetected\\chromedriver.exe' ->
'C:\\Users\\User\\appdata\\roaming\\undetected_chromedriver\\undetected_chromedriver.exe'

Mikayel0 avatar Feb 08 '23 20:02 Mikayel0

Have the same issue. I use this configuration: def init_driver(): chrome_options = uc.ChromeOptions() chrome_options.add_argument("--incognito") chrome_options.add_argument('--headless') chrome_options.add_argument("--log-level=3") chrome_options.add_argument("--start-maximized") driver = uc.Chrome(options=chrome_options, use_subprocess=True) driver.wait = WebDriverWait(driver, 2) return driver

driver = init_driver()

When i use only 1 browser is ok. But when i try to open another

driver_2_example = init_driver()

i get this exception:

  File "C:\Users\Dragur\AppData\Local\Programs\Python\Python39\lib\site-packages\undetected_chromedriver\__init__.py", line 374, in __init__
    if self.patcher.version_main < 108:
TypeError: '<' not supported between instances of 'NoneType' and 'int'

This happens since V110 of Chrome I'm using

undetected-chromedriver   3.4.4
selenium                  4.8.0
Python 3.9.2
Windows 10

Dragur611 avatar Feb 08 '23 20:02 Dragur611

Have the same issue. I use this configuration: def init_driver(): chrome_options = uc.ChromeOptions() chrome_options.add_argument("--incognito") chrome_options.add_argument('--headless') chrome_options.add_argument("--log-level=3") chrome_options.add_argument("--start-maximized") driver = uc.Chrome(options=chrome_options, use_subprocess=True) driver.wait = WebDriverWait(driver, 2) return driver

driver = init_driver()

When i use only 1 browser is ok. But when i try to open another

driver_2_example = init_driver()

i get this exception:

  File "C:\Users\Dragur\AppData\Local\Programs\Python\Python39\lib\site-packages\undetected_chromedriver\__init__.py", line 374, in __init__
    if self.patcher.version_main < 108:
TypeError: '<' not supported between instances of 'NoneType' and 'int'

This happens since V110 of Chrome I'm using

undetected-chromedriver   3.4.4
selenium                  4.8.0
Python 3.9.2
Windows 10

(Yes i'm the same who posted the --headless issue) sorry to bother you so much, I really appreciate your job @ultrafunkamsterdam

Dragur611 avatar Feb 08 '23 20:02 Dragur611

i really shouldnt be doing this.... here we go:

pip install -U https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance

please check if it is really installed, or just run this twice. report back appreciated..

ultrafunkamsterdam avatar Feb 08 '23 22:02 ultrafunkamsterdam

When i tried to start few browsers they started well, after first time i was getting same exception

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:
'C:\\Users\\User\\appdata\\roaming\\undetected_chromedriver\\undetected\\chromedriver.exe' ->
'C:\\Users\\User\\appdata\\roaming\\undetected_chromedriver\\undetected_chromedriver.exe'

Mikayel0 avatar Feb 08 '23 23:02 Mikayel0

https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance

404 error

ERROR: Could not install requirement https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance because of HTTP error 404 Client Error: Not Found for url: https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance for URL https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance

Dragur611 avatar Feb 08 '23 23:02 Dragur611

pip install -U https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance

https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance

404 error

ERROR: Could not install requirement https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance because of HTTP error 404 Client Error: Not Found for url: https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance for URL https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance

please try

pip install -U git+https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance

labodj avatar Feb 09 '23 00:02 labodj

pip install -U https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance

https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance

404 error ERROR: Could not install requirement https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance because of HTTP error 404 Client Error: Not Found for url: https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance for URL https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance

please try

pip install -U git+https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance

Collecting git+https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance
  Cloning https://github.com/ultrafunkamsterdam/undetected-chromedriver (to revision fix-multiple-instance) to c:\users\dragur\appdata\local\temp\pip-req-build-mbcpnq1o
  Running command git clone --filter=blob:none --quiet https://github.com/ultrafunkamsterdam/undetected-chromedriver 'C:\Users\Dragur\AppData\Local\Temp\pip-req-build-mbcpnq1o'
  Running command git checkout -b fix-multiple-instance --track origin/fix-multiple-instance
  branch 'fix-multiple-instance' set up to track 'origin/fix-multiple-instance'.
  Switched to a new branch 'fix-multiple-instance'
  Resolved https://github.com/ultrafunkamsterdam/undetected-chromedriver to commit e146610b7c53beaecf31a31f907c06ef976d1ed5
  Preparing metadata (setup.py) ... done
Requirement already satisfied: selenium>=4.0.0 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from undetected-chromedriver==3.4.5) (4.8.0)
Requirement already satisfied: requests in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from undetected-chromedriver==3.4.5) (2.28.2)      
Requirement already satisfied: websockets in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from undetected-chromedriver==3.4.5) (10.1)      
Requirement already satisfied: trio-websocket~=0.9 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from selenium>=4.0.0->undetected-chromedriver==3.4.5) (0.9.2)   
Requirement already satisfied: trio~=0.17 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from selenium>=4.0.0->undetected-chromedriver==3.4.5) (0.19.0)
Requirement already satisfied: urllib3[socks]~=1.26 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from selenium>=4.0.0->undetected-chromedriver==3.4.5) (1.26.7) 
Requirement already satisfied: certifi>=2021.10.8 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from selenium>=4.0.0->undetected-chromedriver==3.4.5) (2022.12.7)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from requests->undetected-chromedriver==3.4.5) (2.0.9)     
Requirement already satisfied: idna<4,>=2.5 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from requests->undetected-chromedriver==3.4.5) (2.10)
Requirement already satisfied: cffi>=1.14 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from trio~=0.17->selenium>=4.0.0->undetected-chromedriver==3.4.5) (1.15.0)
Requirement already satisfied: outcome in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from trio~=0.17->selenium>=4.0.0->undetected-chromedriver==3.4.5) (1.1.0)
Requirement already satisfied: sniffio in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from trio~=0.17->selenium>=4.0.0->undetected-chromedriver==3.4.5) (1.2.0)
Requirement already satisfied: attrs>=19.2.0 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from trio~=0.17->selenium>=4.0.0->undetected-chromedriver==3.4.5) (21.2.0)
Requirement already satisfied: async-generator>=1.9 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from trio~=0.17->selenium>=4.0.0->undetected-chromedriver==3.4.5) (1.10)
Requirement already satisfied: sortedcontainers in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from trio~=0.17->selenium>=4.0.0->undetected-chromedriver==3.4.5) (2.4.0)
Requirement already satisfied: wsproto>=0.14 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from trio-websocket~=0.9->selenium>=4.0.0->undetected-chromedriver==3.4.5) (1.0.0)
Requirement already satisfied: PySocks!=1.5.7,<2.0,>=1.5.6 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from urllib3[socks]~=1.26->selenium>=4.0.0->undetected-chromedriver==3.4.5) (1.7.1)
Requirement already satisfied: pycparser in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from cffi>=1.14->trio~=0.17->selenium>=4.0.0->undetected-chromedriver==3.4.5) (2.21)
Requirement already satisfied: h11<1,>=0.9.0 in c:\users\dragur\appdata\local\programs\python\python39\lib\site-packages (from wsproto>=0.14->trio-websocket~=0.9->selenium>=4.0.0->undetected-chromedriver==3.4.5) (0.12.0)

I used the command 3 times. But i get same error File "C:\Users\Dragur\AppData\Local\Programs\Python\Python39\lib\site-packages\undetected_chromedriver\__init__.py", line 374, in __init__ if self.patcher.version_main < 108: TypeError: '<' not supported between instances of 'NoneType' and 'int'

Dragur611 avatar Feb 09 '23 00:02 Dragur611

i really shouldnt be doing this.... here we go:

pip install -U https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance

please check if it is really installed, or just run this twice. report back appreciated..

tried that, but get same error

Dragur611 avatar Feb 09 '23 01:02 Dragur611

I have the same issue, any solution would be appreciated!

tomslarkin avatar Feb 09 '23 04:02 tomslarkin

I may have figured out a workaround just now. in the "patcher.py" file I added "import secrets" at the top and then changed: from: prefix = 'undetected'

to: prefix = secrets.token_hex(8)

I figured this out by looking back at version 3.2.1 of undetected-chromedriver, and that's how it was working before.

tomslarkin avatar Feb 09 '23 04:02 tomslarkin

I may have figured out a workaround just now. in the "patcher.py" file I added "import secrets" at the top and then changed: from: prefix = 'undetected'

to: prefix = secrets.token_hex(8)

I figured this out by looking back at version 3.2.1 of undetected-chromedriver, and that's how it was working before.

Worked for me! Thanks!

Dragur611 avatar Feb 09 '23 17:02 Dragur611

do this: chrome_driver = uc.Chrome(version_main=CHROME_VERSION, options=chrome_options)

will fix the issue

vsuyvkpzica46088 avatar Feb 10 '23 02:02 vsuyvkpzica46088

do this: chrome_driver = uc.Chrome(version_main=CHROME_VERSION, options=chrome_options)

will fix the issue

No, it doesn't

Anyway... I just retested the code updated to 57a2bd3f436832ffe8b127efa1d358a64f32be6f and it seems to be working again using chrome 110.

You can try it like so

python -m pip uninstall undetected-chromedriver
python -m pip install git+https://github.com/ultrafunkamsterdam/undetected-chromedriver@fix-multiple-instance
python -m pip install --upgrade selenium

labodj avatar Feb 10 '23 16:02 labodj

I may have figured out a workaround just now. in the "patcher.py" file I added "import secrets" at the top and then changed: from: prefix = 'undetected'

to: prefix = secrets.token_hex(8)

I figured this out by looking back at version 3.2.1 of undetected-chromedriver, and that's how it was working before.

Thaaaaaaaank you. it's worked for me.

abd0014 avatar Feb 20 '23 20:02 abd0014

My two browsers are opening but I see this issue: Only the first one is making the requests and what i have programmed. The second one stays only in the initial chrome screen.

I'm using different profiles in each driver!!!! So i set --profile-directory and --user-data-dir

selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:51381

FIXED=

Split your profile data under diferent folders, dont use same one, one per eaceh

antooro avatar Feb 22 '23 12:02 antooro

My two browsers are opening but I see this issue: Only the first one is making the requests and what i have programmed. The second one stays only in the initial chrome screen.

selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:51381

try to update chrome from settings to the latest version

abd0014 avatar Feb 22 '23 12:02 abd0014

Everything is updated

antooro avatar Feb 22 '23 12:02 antooro

Hi antooro, did you fix the issue mentioned? are you able to manage every single instance correctly?

MaverickAlonso avatar Mar 06 '23 22:03 MaverickAlonso

I tried running this branch with Chrome 111 and Incapsula will trip on it. When I switch back to the regular version it works just fine.

PatrickVdWillik avatar Mar 12 '23 21:03 PatrickVdWillik

I may have figured out a workaround just now. in the "patcher.py" file I added "import secrets" at the top and then changed: from: prefix = 'undetected'

to: prefix = secrets.token_hex(8)

I figured this out by looking back at version 3.2.1 of undetected-chromedriver, and that's how it was working before.

Worked for me too, thanks!

Spykerwolf avatar Mar 17 '23 05:03 Spykerwolf

I noticed this issue as well. Latest changes added prevent running multiple instances of the driver at the same time because the naming was changed from random 8 numbers to fixed string "undetected". The result of this is that only 1 file can exist and an error is thrown if a second file is created. I did not dig further into the code, but do we even need multiple executable at all? If not then it looks like detection if the executable already exists is broken, which would make this a bug.

simplexx avatar Mar 25 '23 16:03 simplexx

Instead of making it yourselves hard, mind you there is a branch for this ....

ultrafunkamsterdam avatar Mar 26 '23 00:03 ultrafunkamsterdam

EDIT: Please ignore below. Seems like I f'ed up something during testing (probably used unmodified driver in the test by mistake). Currently trying to run it on my server (ARM architecture on AWS). But it seems like ARM is not supported at the moment. Driver won't start throwing a wrong executable type error :(

Now I am sure it's a bug, because it happened on my win 10 machine, but when i tried it on ubuntu 20 on the server there was no such issue. So windows (10?) specific bug. In both cases I run selenium headless btw. In any case nice project! Thanks for maintaining it!

simplexx avatar Mar 26 '23 10:03 simplexx

So I have created a new server with a supported architecture, and these are the results: New branch: Does not work, throws the following error: /usr/bin/chromium-browser: 12: xdg-settings: not found /usr/bin/chromium-browser: 12: xdg-settings: not found [9126:9300:0326/154912.698727:ERROR:object_proxy.cc(623)] Failed to call method: org.freedesktop.DBus.ListActivatableNames: object_path= /org/freedesktop/DBus: org.freedesktop.DBus.Error.AccessDenied: An AppArmor policy prevents this sender from sending this message to this recipient; type="method_call", sender=":1.43" (uid=1000 pid=9126 comm="/snap/chromium/2406/usr/lib/chromium-browser/chrom" label="snap.chromium.chromium (enforce)") interface="org.freedesktop.DBus" member="ListActivatableNames" error name="(unset)" requested_reply="0" destination="org.freedesktop.DBus" (bus) [9126:9126:0326/154912.705819:ERROR:process_singleton_posix.cc(334)] Failed to create /home/ubuntu/.local/share/undetected_chromedriver/tmpwymqvrgk/SingletonLock: Permission denied (13) [9120:9308:0326/154912.724564:ERROR:object_proxy.cc(623)] Failed to call method: org.freedesktop.DBus.ListActivatableNames: object_path= /org/freedesktop/DBus: org.freedesktop.DBus.Error.AccessDenied: An AppArmor policy prevents this sender from sending this message to this recipient; type="method_call", sender=":1.44" (uid=1000 pid=9120 comm="/snap/chromium/2406/usr/lib/chromium-browser/chrom" label="snap.chromium.chromium (enforce)") interface="org.freedesktop.DBus" member="ListActivatableNames" error name="(unset)" requested_reply="0" destination="org.freedesktop.DBus" (bus) [9120:9120:0326/154912.747078:ERROR:process_singleton_posix.cc(334)] Failed to create /home/ubuntu/.local/share/undetected_chromedriver/tmp5d0tx2fo/SingletonLock: Permission denied (13) [9325:9325:0326/154912.760216:ERROR:broker_posix.cc(43)] Invalid node channel message [9324:9324:0100/000000.761303:ERROR:broker_posix.cc(43)] Invalid node channel message [9341:9341:0100/000000.797590:ERROR:broker_posix.cc(43)] Invalid node channel message [9354:9354:0326/154912.812891:ERROR:broker_posix.cc(43)] Invalid node channel message

This solution provided by @tomslarkin works:

I may have figured out a workaround just now. in the "patcher.py" file I added "import secrets" at the top and then changed: from: prefix = 'undetected'

to: prefix = secrets.token_hex(8)

I figured this out by looking back at version 3.2.1 of undetected-chromedriver, and that's how it was working before.

simplexx avatar Mar 26 '23 15:03 simplexx

I may have figured out a workaround just now. in the "patcher.py" file I added "import secrets" at the top and then changed: from: prefix = 'undetected'

to: prefix = secrets.token_hex(8)

I figured this out by looking back at version 3.2.1 of undetected-chromedriver, and that's how it was working before.

Thanks, it worked!

rft0 avatar Mar 30 '23 12:03 rft0

"I seem to be having trouble locating the 'patcher.py' file. Would someone be able to assist me in finding it? Any help would be greatly appreciated. Thank you!

mohamed2003-kill avatar Apr 15 '23 23:04 mohamed2003-kill

"I seem to be having trouble locating the 'patcher.py' file. Would someone be able to assist me in finding it? Any help would be greatly appreciated. Thank you!

same problem , we need a solution asap

tatakaetatakaetatakae avatar Apr 15 '23 23:04 tatakaetatakaetatakae

I may have figured out a workaround just now. in the "patcher.py" file I added "import secrets" at the top and then changed: from: prefix = 'undetected' to: prefix = secrets.token_hex(8) I figured this out by looking back at version 3.2.1 of undetected-chromedriver, and that's how it was working before.

It worked. Thank you

GinVlad avatar Apr 17 '23 10:04 GinVlad

"I seem to be having trouble locating the 'patcher.py' file. Would someone be able to assist me in finding it? Any help would be greatly appreciated. Thank you!

same problem , we need a solution asap

First of all, find location of site-packages folder, you can find it with python -m site After that in site-packages folder you need to see the undetected-chromedriver folder.

kker4m avatar Apr 24 '23 17:04 kker4m

I may have figured out a workaround just now. in the "patcher.py" file I added "import secrets" at the top and then changed: from: prefix = 'undetected'

to: prefix = secrets.token_hex(8)

I figured this out by looking back at version 3.2.1 of undetected-chromedriver, and that's how it was working before.

If you still get errors after this solution, you need to set your chrome version in the version_main variable in __init__.py where patcher.py is located. You can do this by using the @vsuyvkpzica46088 method or by giving your Google version to the version_main variable in the __init__.py file manually. As it says in the documentation, you need to give the value ignoring the imperfection (etc: 77.13.0 = 77). The second method works for me.

kker4m avatar Apr 24 '23 18:04 kker4m