I'm using Windows 11 with WSL installed, I've installed the pip version and a few missing libraries to get this for, but Im now stuck with this.
It looks like the the chromedriver crashes with the following " (unknown error: DevToolsActivePort file doesn't exist)"
The only fix I can see is to pass a parameter for headless support. Am I missing something
Full output below
WSLTerminal:~/.wdm/drivers/chromedriver/linux64/114.0.5735.90$ scraperai --url https://www.ycombinator.com/companies
Starting webdriver...
INFO:WDM:====== WebDriver manager ======
INFO:WDM:Get LATEST chromedriver version for google-chrome
INFO:WDM:About to download new driver from https://chromedriver.storage.googleapis.com/114.0.5735.90/chromedriver_linux64.zip
INFO:WDM:Driver downloading response is 200
INFO:WDM:Get LATEST chromedriver version for google-chrome
INFO:WDM:Get LATEST chromedriver version for google-chrome
INFO:WDM:Driver has been saved in cache [/home/steve/.wdm/drivers/chromedriver/linux64/114.0.5735.90]
ERROR:root:Message: unknown error: Chrome failed to start: exited abnormally.
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x5562ab4c84e3
#1 0x5562ab1f7c76
#2 0x5562ab220d78
#3 0x5562ab21d029
#4 0x5562ab25bccc
#5 0x5562ab25b47f
#6 0x5562ab252de3
#7 0x5562ab2282dd
#8 0x5562ab22934e
#9 0x5562ab4883e4
#10 0x5562ab48c3d7
#11 0x5562ab496b20
#12 0x5562ab48d023
#13 0x5562ab45b1aa
#14 0x5562ab4b16b8
#15 0x5562ab4b1847
#16 0x5562ab4c1243
#17 0x7f49b04baac3
Traceback (most recent call last):
File "/home/steve/anaconda3/lib/python3.12/site-packages/scraperai/cli/app.py", line 43, in main
app.run()
File "/home/steve/anaconda3/lib/python3.12/site-packages/scraperai/cli/controller.py", line 258, in run
self.init_crawler()
File "/home/steve/anaconda3/lib/python3.12/site-packages/scraperai/cli/controller.py", line 55, in init_crawler
self.crawler = SeleniumCrawler()
^^^^^^^^^^^^^^^^^
File "/home/steve/anaconda3/lib/python3.12/site-packages/scraperai/crawlers/selenium.py", line 21, in init
self.driver = DefaultChromeWebdriver()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/steve/anaconda3/lib/python3.12/site-packages/scraperai/crawlers/webdriver/local.py", line 36, in init
super().init(service=Service(ChromeDriverManager(driver_version).install()), options=self.options)
File "/home/steve/anaconda3/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 84, in init
super().init(
File "/home/steve/anaconda3/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 104, in init
super().init(
File "/home/steve/anaconda3/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 286, in init
self.start_session(capabilities, browser_profile)
File "/home/steve/anaconda3/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 378, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/steve/anaconda3/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute
self.error_handler.check_response(response)
File "/home/steve/anaconda3/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x5562ab4c84e3
#1 0x5562ab1f7c76
#2 0x5562ab220d78
#3 0x5562ab21d029
#4 0x5562ab25bccc
#5 0x5562ab25b47f
#6 0x5562ab252de3
#7 0x5562ab2282dd
#8 0x5562ab22934e
#9 0x5562ab4883e4
#10 0x5562ab48c3d7
#11 0x5562ab496b20
#12 0x5562ab48d023
#13 0x5562ab45b1aa
#14 0x5562ab4b16b8
#15 0x5562ab4b1847
#16 0x5562ab4c1243
#17 0x7f49b04baac3
ERROR:root:Message: unknown error: Chrome failed to start: exited abnormally.
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x5562ab4c84e3
#1 0x5562ab1f7c76
#2 0x5562ab220d78
#3 0x5562ab21d029
#4 0x5562ab25bccc
#5 0x5562ab25b47f
#6 0x5562ab252de3
#7 0x5562ab2282dd
#8 0x5562ab22934e
#9 0x5562ab4883e4
#10 0x5562ab48c3d7
#11 0x5562ab496b20
#12 0x5562ab48d023
#13 0x5562ab45b1aa
#14 0x5562ab4b16b8
#15 0x5562ab4b1847
#16 0x5562ab4c1243
#17 0x7f49b04baac3
Hi!
I'll check you issue in a few days. Thanks for reporting!
scraperai
