Adrián Chaves
Adrián Chaves
In order to set the current date in the last release notes entry, where we use a ? in unknown digits of the release date, I would like to use...
The [`overwrite` feed option](https://docs.scrapy.org/en/latest/topics/feed-exports.html#feed-options) has a different default value depending on the storage backend used, and we currently cover its default value for all backeds except for [Google Cloud Storage...
This is an undocumented spider attribute that (non-intuitively, in my opinion) overrides the `CONCURRENT_REQUEST_PER_DOMAIN` setting or the `CONCURRENT_REQUEST_PER_IP` setting. If we want to still allow defining a spider attribute that...
When we added support to Scrapy (https://github.com/scrapy/scrapy/pull/2535), we did so without making [brotlipy](https://pypi.org/project/brotlipy/) a hard dependency, seemingly because there seem to be some potential for installation issues. However, given that...
@imrankhan-ik [brought to my attention](https://github.com/scrapy-plugins/scrapy-feedexporter-azure-storage/pull/2#discussion_r932433728) how, in addition to overwriting and appending, users may want to write only if the target does not exist. This seems like the safest default...
While reviewing https://github.com/scrapy/scrapy/pull/4141, I realized the existing `Request.meta` documentation could use some more information.
Imagine you are targeting sites A and B, and you use Splash for _some_ request sent to either. Imagine those sites only support low traffic, so you want to limit...
Cover how to identify a bottleneck (network input/output, CPU, disk I/O, memory) and how to improve Scrapy’s performance for any given bottleneck. Cover `PYTHONOPTIMIZE` where relevant, but: - Explain that...
We provide a [list of special metadata keys](https://docs.scrapy.org/en/latest/topics/request-response.html#topics-request-meta) in the documentation, which are the metadata keys that may be used to interact with built-in Scrapy components. It would be useful...
Fixes #1225, closes #5793. The approach is based on @immerrr’s https://github.com/scrapy/scrapy/issues/1225#issuecomment-339613236. However, because request fingerprinting takes into account information beyond the URL, I created a new meta key just for...