Alexander Sibiryakov

Good catch @clarksun! I'm going to merge it.

I have checked this code again, @clarksun and found that SW was already running the flush when stopping https://github.com/scrapinghub/frontera/blob/master/frontera/worker/strategy.py#L291, if we apply your patch it will be flushing two times....

I think first step for @m-usman-dar is make Scrapy work without Frontera with Splash using SplashMiddleware.

Hi @International , good that you managed to solve it yourself. Body is absent in request because of historical reasons. I think it should be there. Please submit a PR.

Yes, cityhash can't compile on Windows. Honestly, we (Scrapinghub) never tried Frontera on Windows, so likely you will run into previously undiscovered issues. Therefore, I'm suggesting to switch to Linux....

This is definitely related to your connection of SW to Thrift. I'm suggesting to evaluate settings for Thrift server closely and also look at it's load. IMO, it closes connection...

Luiz, It's quite likely connected with the thrift server protocol and transport type. Try it with non-blocking transport and compact protocol. When you start server use the options: -compact and...

> Currently frontera passes an URL to URL_FINGERPRINT_FUNCTION which is already canonicalized by w3lib's canonicalize_url function Only if URL comes from Scrapy's link extractor with canonicalisation enabled. Also there is...

Which error?! Please put the whole stack trace including exception here.

@isra17 thanks for the contribution! we don't have irc or other chat channel, because there is not that big demand. I'm not sure I understand what the problem is: -...