Mikhail Korobov

Hey @raphapassini! Currently the only place CloseSpider is caught is https://github.com/scrapy/scrapy/blob/e45ef7dcd987f00b94e94d71593f6b3664ceb89f/scrapy/core/scraper.py#L151 It seems it needs to be caught in middleware managers as well: spider middleware manager, downloader middleware manager, ExtensionManager...

Currently as a workaround users can use CONCURRENT_REQUESTS_... options; it should give similar results (see https://docs.scrapy.org/en/latest/topics/autothrottle.html#how-it-works), but not exactly the same.

@wRAR Let's try to focus on how Case 2 (or any of these cases) affect https://github.com/zytedata/zyte-spider-templates, not on the case itself. The priority of supporting meta is not clear to...

Allowing andi to accept ``{"arg_1": Engine, "arg_2": Wheels, "arg_3": BikeRack}`` dicts instead of callables or classes makes sense to me; +1 to implement this. I'd probably avoid calling it "signature"...

@mikekutzma yes.

Having cookie management builtin makes more sense to me. Of course, nothing prevents creating a separate library for that (well, maybe https://github.com/scrapy/scrapy/issues/1877 can be a problem), but I'd prefer having...

https://github.com/scrapy/scrapy/pull/3563#issuecomment-505105007 has yet another syntax proposal (haven't thought about it in depth though).

+1 to add an implementation of https://mimesniff.spec.whatwg.org. Repeating https://github.com/scrapy/scrapy/pull/2393#issuecomment-284548599, currently the logic for detecting response type is different from what browsers do; Scrapy is inconsistent and incomplete in mime sniffing...

@shner-elmo there are some more caveats, even unrelated to JS; see https://docs.scrapy.org/en/latest/topics/developer-tools.html#caveats-with-inspecting-the-live-browser-dom