scrapy-selenium issues

Results 79 scrapy-selenium issues

Sort by recently updated

Json API Response's content-type is text/html;charset=UTF-8 by I want application/json;

crawling vuejs site' background json data api 1. The Request: ``` yield SeleniumRequest( url=json_api_url, wait_time=3, callback=self.parse_api) ``` 2. The Origin Response: ``` {"data":{"list":[{"title":"adidas originals Yeezy 450 "Cloud White" H68038"},{"title":"adidas "Have...

workji

Does the `wait_time` argument need a `wait_until` to work correctly?

In the following parser I want the spider to SeleniumRequest all links on a page according to the rules I have specified in the Srapy LinkExtractor 'le'. It seems to...

tungland

Updated 'options' parameter from deprecated values.

Updated 'options' parameter: chrome_ options and firefox_options are deprecated. Check Selenium documentation https://www.selenium.dev/documentation/webdriver/getting_started/open_browser/

ferromauro

Run Scrapy with more than 1 browser.

Hello, There are any ideas how to modify Selenium Middleware to separate ongoing requests in several browsers window?

borys25ol

How to deal with webdriver versioning issues?

I'm getting `selenium.common.exceptions.SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 88 Current browser version is 93.0.4577.63 with binary path /usr/bin/google-chrome` and I can't find a...

smileservices

selenium script written in _init_ function not executing

code : ``` import scrapy from scrapy.selector import Selector from selenium import webdriver from selenium.webdriver.chrome.options import Options from shutil import which class test_2(scrapy.Spider): name='test_2' #allowed_domains=[] start_urls=[ 'https://www.jackjones.in/st-search?q=shoes' ] def _init_(self):...

MADDY312

about proxy support

Good day! Whether there was any solution in support of proxy servers after all? Unfortunately, the standard expression request.meta['proxy'] does not work with SeleniumRequest. Dear zhangtemplar apparently also failed to...

free01man

enhancement

Chrome headless mode isn't working

I followed the instruction to edit settings.py as : SELENIUM_DRIVER_NAME = 'chrome' SELENIUM_DRIVER_EXECUTABLE_PATH = '/Users/****/chromedriver' SELENIUM_DRIVER_ARGUMENTS=['--headless'] However, the Chrome will be launched every time I run the spider. BTW, with...

shengyilu