Adrián Chaves
Adrián Chaves
> It also means that size of responses stored in `info.downloaded` is not expected to exceed value of `SCRAPER_SLOT_MAX_ACTIVE_SIZE` setting. It is my understanding that `info.downloaded` contains every (media pipeline,...
Related: https://github.com/scrapy/scrapy/pull/5566
There’s already [a pull request for it](https://github.com/scrapy/scrapy/pull/4995), and this is a change that will not be merged for a while, so while it is still open, there’s no more work...
Could you add a couple of tests, one for regular raising and another one for raising of a subclass? I think we may also want to suggest the use of...
It sounds like a bug to me. I plan to try and reproduce it when I get some spare time.
After looking into it, I’ve identified that the problem is in https://github.com/scrapy/scrapy/blob/0e579182319504ba7bfd0c09333fe92f70c6d312/scrapy/core/downloader/__init__.py#L135-L147 , `slot.lastseen` is a single value, where maybe it should be a deque that rotates with the enqueued...
We should make sure https://github.com/scrapy/scrapy/issues/5083#issuecomment-824100114 is clear in the documentation of `DOWNLOAD_DELAY`, this is bound to be a common misunderstanding.
Anyone willing to translate the Scrapy documentation once it is possible, please also consider [working on a Scrapy article for the Wikipedia in your language](https://github.com/scrapy/scrapy/issues/4233).
Before translations can happen, [internationalization needs to happen](https://github.com/scrapy/scrapy/pull/4399).
There seems to be already such a project: https://pypi.org/project/mimesniff/