aiohttp_chromium
aiohttp_chromium copied to clipboard
Published
3 months ago
•
milahu
milahu
`selenium_driverless` causing multiple errors
First this happened:
> python /main.py
2024-09-11 18:09:15,798 asyncio DEBUG Using proactor: IocpProactor
2024-09-11 18:09:15,805 aiohttp_chromium.client DEBUG ClientSession: init
2024-09-11 18:09:15,807 aiohttp_chromium.client DEBUG ClientSession: using tempdir AppData\Local\Temp\aiohttp_chromium.ujsh0v6_
2024-09-11 18:09:15,808 aiohttp_chromium.client DEBUG ClientSession: starting chromium
2024-09-11 18:09:15,809 aiohttp_chromium.client DEBUG ClientSession: using chromium user-data-dir AppData\Local\Temp\aiohttp_chromium.ujsh0v6_/chromium-user-data
2024-09-11 18:09:15,825 Ublock DEBUG init done: path: AppData\Local\Temp\aiohttp_chromium.ujsh0v6_/extensions/ublock id: fopomajdelnjnmegngdicnlhnngapano
2024-09-11 18:09:16,302 Ublock DEBUG patch: patching /js/background.js
2024-09-11 18:09:16,303 Ublock DEBUG async_init done
2024-09-11 18:09:16,304 NoDownloadShelf DEBUG init done: path: AppData\Local\Temp\aiohttp_chromium.ujsh0v6_/extensions/no_download_shelf id: eliahhpaikhblhffngfbbgibhmnjplnk
2024-09-11 18:09:16,309 NoDownloadShelf DEBUG async_init done
2024-09-11 18:09:16,310 ClearDownloads DEBUG init done: path: AppData\Local\Temp\aiohttp_chromium.ujsh0v6_/extensions/clear_downloads id: odjhdehjficfhidfnhdgiomhjlfdpddi
2024-09-11 18:09:16,319 ClearDownloads DEBUG async_init done
2024-09-11 18:09:16,319 aiohttp_chromium.client DEBUG writing chromium config file: AppData\Local\Temp\aiohttp_chromium.ujsh0v6_/chromium-user-data/Default/Preferences
2024-09-11 18:09:16,320 aiohttp_chromium.client DEBUG writing chromium state file: AppData\Local\Temp\aiohttp_chromium.ujsh0v6_/chromium-user-data/Local State
2024-09-11 18:09:16,322 aiohttp_chromium.client DEBUG _start_chromium: selenium_webdriver.Chrome
Traceback (most recent call last):
File "main.py", line 523, in <module>
asyncio.run(main())
File "AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "main.py", line 509, in main
async with YahooShopping(
File "main.py", line 166, in __aenter__
await self.session.__aenter__()
File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 940, in __aenter__
return await self
^^^^^^^^^^
File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 950, in _async_init
await self._start_chromium()
File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 1206, in _start_chromium
driver = await selenium_webdriver.Chrome(**chromium_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv\Lib\site-packages\selenium_driverless\webdriver.py", line 114, in __init__
if not options.binary_location:
^^^^^^^^^^^^^^^^^^^^^^^
File "venv\Lib\site-packages\selenium_driverless\types\options.py", line 285, in binary_location
self._binary_location = find_chrome_executable()
^^^^^^^^^^^^^^^^^^^^^^^^
File "venv\Lib\site-packages\selenium_driverless\utils\utils.py", line 67, in find_chrome_executable
raise FileNotFoundError("Couldn't find installed Chrome or Chromium executable")
FileNotFoundError: Couldn't find installed Chrome or Chromium executable
Therefore I changed the code that calls find_chrome_executable() and specified the path manually. Then:
> python /main.py
2024-09-11 18:04:58,841 asyncio DEBUG Using proactor: IocpProactor
2024-09-11 18:04:58,850 aiohttp_chromium.client DEBUG ClientSession: init
2024-09-11 18:04:58,852 aiohttp_chromium.client DEBUG ClientSession: using tempdir AppData\Local\Temp\aiohttp_chromium.2bugab3j
2024-09-11 18:04:58,854 aiohttp_chromium.client DEBUG ClientSession: starting chromium
2024-09-11 18:04:58,855 aiohttp_chromium.client DEBUG ClientSession: using chromium user-data-dir AppData\Local\Temp\aiohttp_chromium.2bugab3j/chromium-user-data
2024-09-11 18:04:58,875 Ublock DEBUG init done: path: AppData\Local\Temp\aiohttp_chromium.2bugab3j/extensions/ublock id: eobhiineohjdbcoilbgglmjngkiidalo
downloading https://github.com/gorhill/uBlock/releases/download/1.55.0/uBlock0_1.55.0.chromium.zip
200
2024-09-11 18:05:00,498 Ublock DEBUG patch: patching /js/background.js
2024-09-11 18:05:00,499 Ublock DEBUG async_init done
2024-09-11 18:05:00,499 NoDownloadShelf DEBUG init done: path: AppData\Local\Temp\aiohttp_chromium.2bugab3j/extensions/no_download_shelf id: nlomidnifpnncoganambdflkifgbabmi
downloading https://github.com/aoirint/NoDownloadShelf/archive/342bbc883e276cdbcb0dee305fcb7c0bc7cb56b5.zip
200
2024-09-11 18:05:01,182 NoDownloadShelf DEBUG async_init done
2024-09-11 18:05:01,183 ClearDownloads DEBUG init done: path: AppData\Local\Temp\aiohttp_chromium.2bugab3j/extensions/clear_downloads id: fegandgcgdakafpfflbleamnpcnkjlgb
downloading https://github.com/sindresorhus/clear-downloads/archive/35c58beb63256eb6f8fbe55cd4a11ebe51fb044c.zip
200
2024-09-11 18:05:01,910 ClearDownloads DEBUG async_init done
2024-09-11 18:05:01,910 aiohttp_chromium.client DEBUG writing chromium config file: AppData\Local\Temp\aiohttp_chromium.2bugab3j/chromium-user-data/Default/Preferences
2024-09-11 18:05:01,912 aiohttp_chromium.client DEBUG writing chromium state file: AppData\Local\Temp\aiohttp_chromium.2bugab3j/chromium-user-data/Local State
2024-09-11 18:05:01,913 aiohttp_chromium.client DEBUG _start_chromium: selenium_webdriver.Chrome
Traceback (most recent call last):
File "venv\Lib\site-packages\aiohttp\connector.py", line 1073, in _wrap_create_connection
sock = await aiohappyeyeballs.start_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv\Lib\site-packages\aiohappyeyeballs\impl.py", line 104, in start_connection
raise first_exception
File "venv\Lib\site-packages\aiohappyeyeballs\impl.py", line 81, in start_connection
sock = await _connect_sock(
^^^^^^^^^^^^^^^^^^^^
File "venv\Lib\site-packages\aiohappyeyeballs\impl.py", line 166, in _connect_sock
await loop.sock_connect(sock, address)
File "AppData\Local\Programs\Python\Python311\Lib\asyncio\proactor_events.py", line 726, in sock_connect
return await self._proactor.connect(sock, address)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "AppData\Local\Programs\Python\Python311\Lib\asyncio\windows_events.py", line 846, in _poll
value = callback(transferred, key, ov)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "AppData\Local\Programs\Python\Python311\Lib\asyncio\windows_events.py", line 633, in finish_connect
ov.getresult()
ConnectionRefusedError: [WinError 1225] リモート コンピューターによりネットワーク接続が拒否されました。
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "venv\Lib\site-packages\selenium_driverless\types\base_target.py", line 76, in _init
res = await session.get(url, timeout=10)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv\Lib\site-packages\aiohttp\client.py", line 657, in _request
conn = await self._connector.connect(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv\Lib\site-packages\aiohttp\connector.py", line 564, in connect
proto = await self._create_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv\Lib\site-packages\aiohttp\connector.py", line 975, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv\Lib\site-packages\aiohttp\connector.py", line 1350, in _create_direct_connection
raise last_exc
File "venv\Lib\site-packages\aiohttp\connector.py", line 1319, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv\Lib\site-packages\aiohttp\connector.py", line 1088, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 127.0.0.1:54112 ssl:default [リモート コンピューターによりネットワーク接続が拒否されました。]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main.py", line 523, in <module>
asyncio.run(main())
File "AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "main.py", line 509, in main
async with YahooShopping(
File "main.py", line 166, in __aenter__
await self.session.__aenter__()
File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 940, in __aenter__
return await self
^^^^^^^^^^
File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 950, in _async_init
await self._start_chromium()
File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 1206, in _start_chromium
driver = await selenium_webdriver.Chrome(**chromium_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv\Lib\site-packages\selenium_driverless\webdriver.py", line 260, in start_session
self._base_target = await BaseTarget(host=self._host, is_remote=self._is_remote,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "venv\Lib\site-packages\selenium_driverless\types\base_target.py", line 81, in _init
raise asyncio.TimeoutError(
TimeoutError: Couldn't connect to chrome within 30 seconds
This is odd because the following tutorial provided by the selenium_driverless docs worked fine at least until line 11 where TimeoutError is raised.
from selenium_driverless import webdriver
from selenium_driverless.types.by import By
import asyncio
async def main():
options = webdriver.ChromeOptions()
async with webdriver.Chrome(options=options) as driver:
await driver.get('http://nowsecure.nl#relax', wait_load=True)
await driver.sleep(0.5)
await driver.wait_for_cdp("Page.domContentEventFired", timeout=15)
# wait 10s for elem to exist
elem = await driver.find_element(By.XPATH, '/html/body/div[2]/div/main/p[2]/a', timeout=10)
await elem.click(move_to=True)
alert = await driver.switch_to.alert
print(alert.text)
await alert.accept()
print(await driver.title)
asyncio.run(main())
OS: Windows 10, Python: 3.11 Thanks in advance
File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 1206, in _start_chromium
driver = await selenium_webdriver.Chrome(**chromium_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
TimeoutError: Couldn't connect to chrome within 30 seconds
seems like some of chromium_args make chrome start too slow
try to raise the timeout to 99999 seconds
try to remove some of chromium_args
see also: upstream issue https://github.com/kaliiiiiiiiii/Selenium-Driverless/issues/214
I removed **chromium_args on line 1206 in client.py, but still the same problem persists. I'm sorry, but I'm running out of time, so I give up this.
Instantiating Chrome before using this library (and after importing this library) does not cause the error, so it is likely that some global settings change after the instantiation of ClientSession to be the cause.