scrapy-selenium icon indicating copy to clipboard operation
scrapy-selenium copied to clipboard

The key not found error shows up

Open JDRanpariya opened this issue 4 years ago • 7 comments

I started using this module first it shows yellow underline in vscode under module name and when i print response.meta dict it contains only following: {'download_timeout': 180.0, 'download_slot': 'golden.com', 'download_latency': 0.34168577194213867} it doesn't contain screenshot or driver etc. what's the problem

JDRanpariya avatar May 02 '20 05:05 JDRanpariya

+1, still same

boardsofcanada avatar Jul 13 '20 09:07 boardsofcanada

Same issue here, @JDRanpariya , @boardsofcanada any clue ?

tristanlatr avatar Aug 25 '20 23:08 tristanlatr

The issue was in my Selenium Contract tests. Testing by default yields normal Scrapy.Request objects. See source code here https://docs.scrapy.org/en/latest/_modules/scrapy/contracts.html

In order to workaround this and still use contracts to test ou spiders we need to create a new contract

from scrapy_selenium import SeleniumRequest
class WithSelenium(Contract):
    """ Contract to set the request class to be SeleniumRequest for the current call back method to test

    @with_selenium
    """
    name = 'with_selenium'
    request_cls = SeleniumRequest

Then use the contract @with_selenium in our testing. Hope it helps

tristanlatr avatar Aug 26 '20 15:08 tristanlatr

+1 Still the same, any updates!

mhamedLmarbouh avatar Oct 27 '20 10:10 mhamedLmarbouh

+1

SteveSmirnoff avatar Nov 03 '20 01:11 SteveSmirnoff

for some reason when I copied the code for SeleniumRequest, SeleniumMiddelware to my project it all worked just fine!

mhamedLmarbouh avatar Nov 03 '20 10:11 mhamedLmarbouh

Had the same issue before, for me, it's solved after install the driver (chromedriver or geckodriver, because the missing of driver made the selenium middleware disabled)

Lpaydat avatar Jan 28 '21 06:01 Lpaydat