Dharmesh Pandav

TypeError: proxy_check() missing 1 required positional argument: 'proxy'

8

I have freshly installed GoogleScraper and using it with Proxy file Have changed following config file parameter : ``` Python #config.cfg file keywords: filetype:pdf pwn2own use_own_ip: False check_proxies: True proxy_file:proxyfile.txt...

Replace deprecated request_fingerprint with fingerprint in cache and dupefilter files

1

`request_fingerprint` is deprecated in scrapy 2.7 and is replaced by `fingerprint`, Using `request_fingerprint` results in deprecation warning in logs https://docs.scrapy.org/en/latest/topics/request-response.html#std-setting-REQUEST_FINGERPRINTER_IMPLEMENTATION https://github.com/scrapy-plugins/scrapy-splash/blob/6a9eb9cd35bc81bebec9719860da5c8a5d2b35bf/scrapy_splash/cache.py#L13 https://github.com/scrapy-plugins/scrapy-splash/blob/6a9eb9cd35bc81bebec9719860da5c8a5d2b35bf/scrapy_splash/dupefilter.py#L20 https://github.com/scrapy-plugins/scrapy-splash/blob/6a9eb9cd35bc81bebec9719860da5c8a5d2b35bf/scrapy_splash/__init__.py#L10

ValueTooLarge: Value exceeds max encoded size of 1048576 bytes

2

the issue arises when there are null characters ( \x00) present in the response.body for len(response.body) == every null characters will be of length 1 but when we are encoding...

Possible improvement: this check depends on scrapy response properly converted to XmlResponse instance

3

This check here depends on other modules and relies on if scrapy response had been converted to an instance of "XmlResponse" https://github.com/scrapy/scrapy/blob/b88f22c6c5de4ca8828b2abe860516c2468c4fe3/scrapy/spiders/sitemap.py#L109 The assumption here that the response will be...