scrapy-splash icon indicating copy to clipboard operation
scrapy-splash copied to clipboard

scrapy_splash/dupefilter.py:20: ScrapyDeprecationWarning: Call to deprecated function scrapy.utils.request.request_fingerprint().

Open htch opened this issue 1 year ago • 1 comments

Scrapy throws a deprecation warning

python3.9/site-packages/scrapy_splash/dupefilter.py:20: ScrapyDeprecationWarning: Call to deprecated function scrapy.utils.request.request_fingerprint().

    If you are using this function in a Scrapy component, and you are OK with users of your component changing the fingerprinting algorithm through settings, use crawler.request_fingerprinter.fingerprint() instead in your Scrapy component (you can get the crawler object from the 'from_crawler' class method).

     Otherwise, consider using the scrapy.utils.request.fingerprint() function instead.                                                      
                                                                                                                                            
     Either way, the resulting fingerprints will be returned as bytes, not as a string, and they will also be different from those generated by 'request_fingerprint()'. Before you switch, make sure that you understand the consequences of this (e.g. cache invalidation) and are OK with them; otherwise, consider implementing your own function which returns the same fingerprints as the deprecated 'request_fingerprint()' function.
      fp = request_fingerprint(request, include_headers=include_headers)
                                                                                                                                            
     python3.9/site-packages/scrapy_splash/dupefilter.py:20: ScrapyDeprecationWarning: Call to deprecated function scrapy.utils.request.request_fingerprint().                                                                                                                         

    If you are using this function in a Scrapy component, and you are OK with users of your component changing the fingerprinting algorithm through settings, use crawler.request_fingerprinter.fingerprint() instead in your Scrapy component (you can get the crawler object from the 'from_crawler' class method).

    Otherwise, consider using the scrapy.utils.request.fingerprint() function instead.                                                      
                                                                                                                                             
     Either way, the resulting fingerprints will be returned as bytes, not as a string, and they will also be different from those generated by 'request_fingerprint()'. Before you switch, make sure that you understand the consequences of this (e.g. cache invalidation) and are OK with them; otherwise, consider implementing your own function which returns the same fingerprints as the deprecated 'request_fingerprint()' function.
       fp = request_fingerprint(request, include_headers=include_headers)

htch avatar Sep 11 '24 18:09 htch

2024-10-20 14:54:55 [py.warnings] WARNING: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/scrapy_splash/dupefilter.py:20: ScrapyDeprecationWarning: Call to deprecated function scrapy.utils.request.request_fingerprint().

If you are using this function in a Scrapy component, and you are OK with users of your component changing the fingerprinting algorithm through settings, use crawler.request_fingerprinter.fingerprint() instead in your Scrapy component (you can get the crawler object from the 'from_crawler' class method).

Otherwise, consider using the scrapy.utils.request.fingerprint() function instead.

Either way, the resulting fingerprints will be returned as bytes, not as a string, and they will also be different from those generated by 'request_fingerprint()'. Before you switch, make sure that you understand the consequences of this (e.g. cache invalidation) and are OK with them; otherwise, consider implementing your own function which returns the same fingerprints as the deprecated 'request_fingerprint()' function. fp = request_fingerprint(request, include_headers=include_headers)

biaobro avatar Oct 20 '24 06:10 biaobro

This should be fixed by https://github.com/scrapy-plugins/scrapy-splash/pull/317

wRAR avatar Jan 24 '25 12:01 wRAR