scrapy-selenium

scrapy-selenium copied to clipboard

Hi Clemenet. I am just integrating the code into my work. My first part is to log in (already done with only Scrapy). I don't know how to run it...

crianopa

Implicitly Wait

1

Hi I see that you've implemented Explicit wait of Selenium. I wanted to ask you why you didn't implement Implicit Wait, is there a problem you see to implement that?...

Triplemars

Simulate a click then wait

3

I want to be able to run my script command then wait for `n` number of seconds then get the page content. This is actually to simulate a `next page`...

markronquillo

Allow users to set Firefox preferences via scrapy settings. This provides a generic way to set cookie lifetimes, proxies, etc. ```python SELENIUM_DRIVER_PREFERENCES={ 'network.proxy.type': 1, 'network.proxy.socks_version': 5, 'network.proxy.socks': 'localhost', 'network.proxy.socks_port': 9000,...

phaer

Hi @clemfromspace I'm trying to install scrapy-selenium using the *setup.py* file (to enjoy the possibility of use a remote selenium operation), but it fails if I have pip version greater...

michelts

using scrapy FormRequest

1

is there any way to use scrapy.FormRequest? like splash have SplashFormRequest, without it what is the option for handling post requests?

expired-brain

Auto-formatting using isort and autopep8

2

The results of: ``` python -m isort -rc --skip venv . python -m autopep8 -r --in-place --exclude venv . ```

motin

Hello, I am trying to use scrapy-selenium with Crawlera but haven't had any luck. Requests are sent, and responses come back, but the Crawlera dashboard doesn't show the requests as...

jlim782

TypeError: WebDriver.__init__() got an unexpected keyword argument 'executable_path'

6

Ubuntu 22.04.3 LTS (Jammy Jellyfish) ARM64 Selenium 4.10.0 scrapy-selenium 0.0.7 Mozilla Firefox 115.0.2 geckodriver 0.33.0 ( 2023-07-11) Configured as description, get error TypeError: WebDriver.__init__() got an unexpected keyword argument 'executable_path'...

GANGHSUN

Fix executable_path type error by utilising service when connecting to selenium

1

1. Fixed the executable_path TypeError https://github.com/clemfromspace/scrapy-selenium/issues/128 2. Fixing remote driver functionality by fixing the deprecated desired_capabilities for options. 3. Removed pip_tools when getting the installation requirements for the build in...

malmike