scrapy-selenium
scrapy-selenium copied to clipboard
No Driver meta
I am trying to run scrapy-selenium in an wsl pipenv environment
i have added the
SELENIUM_DRIVER_NAME = 'chrome'
SELENIUM_DRIVER_EXECUTABLE_PATH = '/usr/bin/chromedriver'
SELENIUM_DRIVER_ARGUMENTS=['--headless']
and
DOWNLOADER_MIDDLEWARES = {
'scrapy_selenium.SeleniumMiddleware': 800
}
and SeleniumRequest works just fine, but response.request has no driver meta. running which chromedriver in terminal returns '/usr/bin/chromedriver'
i have tried changing the SELENIUM_DRIVER_EXECUTABLE_PATH to which(chromedriver) and installing chromedriver into my local and direct it there, and i have tried using command_executor and setting browser_executable_path but the same issue persists.
not setting up the driver name, driver path, etc does not create a new error what am i doing wrong?