scrapy-selenium icon indicating copy to clipboard operation
scrapy-selenium copied to clipboard

Using both scrapy.Request and scrapy_selenium.SeleniumRequest in same website session. Is it possible?

Open ghost opened this issue 3 years ago • 0 comments

I was wondering if it is possible to yield requests from both standard Scrapy and scrapy-selenium in the same spider. I have mapped many requests that the website shows in developer tools to compute some values. I think it could save me some requests as the data I need would already have its final value in screen.

In the tests I made, the SeleniumRequest didn't throw any errors, but the response was showing the website login page. Like the session was not being shared.

ghost avatar Dec 15 '20 22:12 ghost