scrapy-splash
scrapy-splash copied to clipboard
scrapy cache not used for splash requests
Hello. I'm using Splash and scrapy-splash. It's quite slow, but it works. My HTTPCACHE_DIR is filled with lot of stuff. I have HTTPCACHE_STORAGE = 'scrapy_splash.SplashAwareFSCacheStorage' as specified in the doc.
Nevertheless, if i restart crawling, it's still very slow, and i can see in splash (docker logs) that all requests are done again.
So basically, my cache is filled but not used. How can I debug this ?