scrapy-deltafetch icon indicating copy to clipboard operation
scrapy-deltafetch copied to clipboard

delta-fetch not scraping new items

Open clpatterson opened this issue 6 years ago • 0 comments

When I run an initial scrape with delta-fetch enabled it tells me that it has stored 263 urls where items have been scraped. I assume when I run it again I should see 263 log lines with info about urls being ignored as well as any newly scraped items, but instead delta-fetch says 14 random urls where ignored and closes. Any suggestions on how to debug this issue would be greatly appreciated!

Please see my stackoverflow question here.

clpatterson avatar Jun 15 '18 20:06 clpatterson