scrapy-selenium
scrapy-selenium copied to clipboard
Chrome headless mode isn't working
I followed the instruction to edit settings.py as :
SELENIUM_DRIVER_NAME = 'chrome' SELENIUM_DRIVER_EXECUTABLE_PATH = '/Users/****/chromedriver' SELENIUM_DRIVER_ARGUMENTS=['--headless']
However, the Chrome will be launched every time I run the spider.
BTW, with only Selenium, headless mode was working correctly:
options = Options()
options.add_argument("--headless")
driver = webdriver.Chrome(service = Service(ChromeDriverManager().install()), options = options)
Does anyone face the same problem? Need some help.
from shutil import which
ChromeDriverManager().install()
also downloads and returns path. so this was working for me
SELENIUM_DRIVER_NAME = 'chrome' SELENIUM_DRIVER_EXECUTABLE_PATH = which(ChromeDriverManager().install()) SELENIUM_DRIVER_ARGUMENTS=['--headless']