scrapy-webdriver icon indicating copy to clipboard operation
scrapy-webdriver copied to clipboard

added the process_spider_exception method in middleware

Open stringertheory opened this issue 11 years ago • 1 comments

To address the issue in Issue #4 of a single exception in a Spider's parse method causing the spider to finish.

stringertheory avatar Jul 12 '13 16:07 stringertheory

Take a look at scrapy_webdriver.tests.test_request_queue. One way to do it would be to base the test on TestRequestQueue.test_priorization, have the mock raise an exception midway, and make sure subsequent requests do get called in addition to somehow test the handling of the exception itself.

ncadou avatar Jul 13 '13 12:07 ncadou