scrapy-selenium

scrapy-selenium copied to clipboard

No usage example

4

No. readme does not count :) The people demand an example! Please provide an example!

rubmz

How to use selenium experimental options in selenium driver arguments

3

from shutil import which SELENIUM_DRIVER_NAME = 'chrome' SELENIUM_DRIVER_EXECUTABLE_PATH = which('chromedriver') prefs = {"profile.managed_default_content_settings.images": 2} SELENIUM_DRIVER_ARGUMENTS=['--incognito',prefs] DOWNLOADER_MIDDLEWARES = { 'scrapy_selenium.SeleniumMiddleware': 800 } here is my code and i want to add...

iampritamraj

Package fails to install with pip 20.x

2

Tried to install directly from git via pip as follows: `-e git://github.com/clemfromspace/scrapy-selenium.git@master#egg=scrapy-selenium"` which fails with pip 20.2 as in the below trace. pip 18.x works fine ``` Traceback (most recent...

alxnik

Update setup.py to accomodate pip 20+

1

This fixes a problem with the newest version of pip: ``` Traceback (most recent call last): File "scrapy-selenium/setup.py", line 5, in from pip.download import PipSession ModuleNotFoundError: No module named 'pip.download'...

Brandl

does not respect DOWNLOAD_DELAY

3

The request is not passed to scrapy downloader, where the DOWNLOAD_DELAY is handle. There is no way to set a delay parameter within this middleware. Is it possible to add...

misssprite

Selenium window size arguments not working

1

Hi guys, thanks for the middleware I have a problem with setting window size , I want to have mobile version of a website but setting user-agent is not enough,...

pitchourou

Hi, thank you for continue work on this project. Do I able to run webdriver with additional options from `selenium.webdriver.firefox.options`? e.g. ``` opts = Options() opts.log.level = "warn" ``` (geckodriver...

sgstq

PyPI release

1

Hello, Nice work with scrapy-selenium! The last PyPI release was over a year ago and that's lacking remote webdriver capabilities etc. Any plans for a release soon? Thanks

aster-anto

SeleniumRequest should use meta to pass arguments

2

SeleniumRequest should use meta to pass arguments ``` self.wait_time = wait_time self.wait_until = wait_until self.screenshot = screenshot self.script = script ``` when use `scrapy_redis.scheduler.Scheduler` that won't be serialized

ixqbar

pip version 20.1.1 does not contain a pip._internal.download module and it changed ParsedRequirements.req to ParsedRequirements.requirements, both of which were used in the setup.py. This broke the ability to install scrapy-selenium....

createdbysk