Chrome w/ Chromedriver 106 - High failure rates of: "InvalidArgumentException: Message: invalid argument: No target with given id found" and "WebDriverException: Message: disconnected: received Inspector.detached event"

[jprouty]

My project (and users) are seeing a new issue which started with the release of chrome & chromedriver v106.

Minimal reproduction case (full code here) - pseudocode:

webdriver = Chrome(options=ChromeOptions(),
                   executable_path=<path to chromedriver>)
webdriver.get('https://www.google.com')
# This fails spuriously:
response = webdriver.request('GET', 'https://www.google.com/images/branding/googlelogo/2x/googlelogo_light_color_272x92dp.png')
response.raise_for_status()

Could you help me triage what's going on? At first glance it seems like the window spawning/window switching isn't working correctly anymore.

I was seeing this on both selenium-requests v2.0.0 and v1.4.1.

Win11:

DevTools listening on ws://127.0.0.1:59212/devtools/browser/fe19f00b-78d2-4317-b61c-47387f710856
[17404:4820:1001/224935.886:ERROR:interface_endpoint_client.cc(689)] Message 0 rejected by interface blink.mojom.WidgetHost
Traceback (most recent call last):
  File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\jeffp\repos\mint-amazon-tagger\mintamazontagger\repro_mac_issue.py", line 32, in <module>
    main()
  File "C:\Users\jeffp\repos\mint-amazon-tagger\mintamazontagger\repro_mac_issue.py", line 21, in main
    response = webdriver.request(
  File "C:\Users\jeffp\repos\mint-amazon-tagger\dev_venv\lib\site-packages\seleniumrequests\request.py", line 159, in request
    self.requests_session.headers = get_webdriver_request_headers(self, proxy_host=self.__proxy_host)
  File "C:\Users\jeffp\repos\mint-amazon-tagger\dev_venv\lib\site-packages\seleniumrequests\request.py", line 76, in get_webdriver_request_headers
    webdriver.switch_to.window(original_window_handle)
  File "C:\Users\jeffp\repos\mint-amazon-tagger\dev_venv\lib\site-packages\selenium\webdriver\remote\switch_to.py", line 134, in window
    self._w3c_window(window_name)
  File "C:\Users\jeffp\repos\mint-amazon-tagger\dev_venv\lib\site-packages\selenium\webdriver\remote\switch_to.py", line 142, in _w3c_window
    send_handle(window_name)
  File "C:\Users\jeffp\repos\mint-amazon-tagger\dev_venv\lib\site-packages\selenium\webdriver\remote\switch_to.py", line 138, in send_handle
    self._driver.execute(Command.SWITCH_TO_WINDOW, {'handle': h})
  File "C:\Users\jeffp\repos\mint-amazon-tagger\dev_venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 429, in execute
    self.error_handler.check_response(response)
  File "C:\Users\jeffp\repos\mint-amazon-tagger\dev_venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: No target with given id found
  (Session info: chrome=106.0.5249.91)
Stacktrace:
Backtrace: Can provide upon request - not useful

Mac:

Traceback (most recent call last):
  File "/usr/local/Cellar/[email protected]/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/Cellar/[email protected]/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/jeff/repos/mint-amazon-tagger/mintamazontagger/repro_selenium_issue.py", line 32, in <module>
    main()
  File "/Users/jeff/repos/mint-amazon-tagger/mintamazontagger/repro_selenium_issue.py", line 21, in main
    response = webdriver.request(
  File "/Users/jeff/repos/mint-amazon-tagger/dev_venv/lib/python3.8/site-packages/seleniumrequests/request.py", line 162, in request
    self.requests_session.headers = get_webdriver_request_headers(
  File "/Users/jeff/repos/mint-amazon-tagger/dev_venv/lib/python3.8/site-packages/seleniumrequests/request.py", line 79, in get_webdriver_request_headers
    webdriver.switch_to.window(original_window_handle)
  File "/Users/jeff/repos/mint-amazon-tagger/dev_venv/lib/python3.8/site-packages/selenium/webdriver/remote/switch_to.py", line 134, in window
    self._w3c_window(window_name)
  File "/Users/jeff/repos/mint-amazon-tagger/dev_venv/lib/python3.8/site-packages/selenium/webdriver/remote/switch_to.py", line 142, in _w3c_window
    send_handle(window_name)
  File "/Users/jeff/repos/mint-amazon-tagger/dev_venv/lib/python3.8/site-packages/selenium/webdriver/remote/switch_to.py", line 138, in send_handle
    self._driver.execute(Command.SWITCH_TO_WINDOW, {'handle': h})
  File "/Users/jeff/repos/mint-amazon-tagger/dev_venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
    self.error_handler.check_response(response)
  File "/Users/jeff/repos/mint-amazon-tagger/dev_venv/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: disconnected: received Inspector.detached event
  (Session info: chrome=106.0.5249.61)

Ubuntu (both errors):

Traceback (most recent call last):
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/bin/mint-amazon-tagger-repro_mac_issue", line 8, in <module>
    sys.exit(main())
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/mintamazontagger/repro_mac_issue.py", line 21, in main
    response = webdriver.request(
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/seleniumrequests/request.py", line 159, in request
    self.requests_session.headers = get_webdriver_request_headers(self, proxy_host=self.__proxy_host)
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/seleniumrequests/request.py", line 76, in get_webdriver_request_headers
    webdriver.switch_to.window(original_window_handle)
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/selenium/webdriver/remote/switch_to.py", line 134, in window
    self._w3c_window(window_name)
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/selenium/webdriver/remote/switch_to.py", line 142, in _w3c_window
    send_handle(window_name)
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/selenium/webdriver/remote/switch_to.py", line 138, in send_handle
    self._driver.execute(Command.SWITCH_TO_WINDOW, {'handle': h})
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
    self.error_handler.check_response(response)
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: disconnected: received Inspector.detached event
  (Session info: chrome=106.0.5249.61)
Stacktrace:
#0 0x5624511412c3 <unknown>
...useless stack...
#34 0x7f65e85b3b43 <unknown>

Traceback (most recent call last):
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/bin/mint-amazon-tagger-repro_mac_issue", line 8, in <module>
    sys.exit(main())
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/mintamazontagger/repro_mac_issue.py", line 21, in main
    response = webdriver.request(
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/seleniumrequests/request.py", line 159, in request
    self.requests_session.headers = get_webdriver_request_headers(self, proxy_host=self.__proxy_host)
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/seleniumrequests/request.py", line 76, in get_webdriver_request_headers
    webdriver.switch_to.window(original_window_handle)
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/selenium/webdriver/remote/switch_to.py", line 134, in window
    self._w3c_window(window_name)
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/selenium/webdriver/remote/switch_to.py", line 142, in _w3c_window
    send_handle(window_name)
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/selenium/webdriver/remote/switch_to.py", line 138, in send_handle
    self._driver.execute(Command.SWITCH_TO_WINDOW, {'handle': h})
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
    self.error_handler.check_response(response)
  File "/home/jeff/repos/mint-amazon-tagger/pypi_test_venv/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: disconnected: received Inspector.detached event
  (failed to check if window was closed: no such frame: Session with given id not found.)
  (Session info: chrome=106.0.5249.61)

[somsak]

I got similar issue on my environment too.

Chrome 106.0.5249.91 + chromedriver mac_arm64 selenium-requests 2.0.0

Somehow I got around by "commenting" all work around for Chrome that thrown exception in request.py

• line 76

#webdriver.switch_to.window(original_window_handle)

• line 163-164

                #while len(self.window_handles) > window_handles_before:
                #    time.sleep(0.01)

That fixed my problem and made my script to work again just fine.

[jprouty]

I can confirm @somsak's workaround as enabling 100% success on Mac. I would like to verify on my win/ubuntu reproduction as well (which is a workstation that replicates nearly 100%).

[jprouty]

Further confirming this workaround works for Win11 and Ubuntu as well. Based on my cursory familiarity, it seems like a chromedriver issue (since accessing webdriver.window_handles seems to cause the detachment).

[cryzed]

Well, I'm not quite sure how to best work around this issue: Chromedriver is being buggy and inconsistent again.

The code in these places is there for a reason, and I'm sure just removing it will lead to different errors down the line. The code in lines 163-164 admittedly can probably be safely removed by now, because the referenced issue seems to have been fixed for a while. However, the code in line 76 will most likely break compatibility with other browsers that have a different default new-tab-switching behavior.

Any suggestions?

[somsak]

For line 76, how about "if Chrome" and check the version? It seems to affecting Chrome version < 106.

[cryzed]

Does this branch work for you? In my testing it affected specifically version 106, and no other version.

[somsak]

Strangely enough, I tried re-installing my seleniumrequests just now (through pip), which got 2.0.0 again. Then I ran my selenium script to confirm that the problem persist. Somehow it was gone.

My chrome version is still the same: 106.0.5249.91

So I didn't test the branch above, but the problem is already vanished for me.

[alpritt]

That branch fixed it on my machine. Also version 106.

[jprouty]

Sorry for the tardy response.

As per @somsak, I tried to replicate the original issue. I was able to do so on Win11 and Ubuntu, but not Mac, using all of the latest stable versions as written here:

Chrome: 106.0.5249.103 (linux) 106.0.5249.119 (win) 106.0.5249.119 (mac) Chromedriver: 106.0.5249.61 Selenium: 4.5.0 Selenium-requests: 2.0

Using @cryzed workaround branch, all 3 platforms do not replicate the issue. I have already pulled in the change to my project as a temporary fix (thus my slow response) until a permanent fix is determined and packaged (via pypi). Thanks all.

[carljm]

The problem still exists in chromedriver/chrome v107. The linked fix branch still works if the version constraint is removed or extended to cover v107 in addition to 106.

[jprouty]

A timely resolution would be appreciated. @cryzed What next steps would you like to see?

[cryzed]

Version 2.0.1 is published on PyPi.

[hudcap]

I'm getting this error with Chrome v108 now...

[cryzed]

I just applied the workaround for v108 in 2.0.2.

[jprouty]

@cryzed I've verified the issue doesn't reproduce on 106-108. Thanks for the fixes thus far.

However, I'm unable to accept the current mechanism of allow listing chrome browser versions just in time as they are marked stable. My project can go indefinitely without need for a release - to do this my app will maintain (and download if needed) the latest stable chromedriver. This means if I cut a release with your package it would presumably stop working once chrome 109 is marked stable (unless chromedriver/chrome reverted the breaking behavior).

Would it be possible to compare chrome versions and look for 106 or larger? Thanks for your consideration.

[cryzed]

The correct fix for this would be to report this upstream to the chromium/chromedriver developers, and fix the actual issue and not just a symptom in some (my) downstream package.

I'll look into parsing the version number and doing this comparison when I find time, it just seems a bit silly that I am constantly working around bugs in webdrivers.

[cryzed]

@jprouty I added your suggestion in v2.0.3, please let me know if it works for you.

[jprouty]

Many thanks!