scrapy-splash icon indicating copy to clipboard operation
scrapy-splash copied to clipboard

Response type still not determined correctly sometimes

Open lopuhin opened this issue 5 years ago • 1 comments

Here is a traceback (without a way to reproduce unfortunately), which shows that most likely we still sometimes don't identify response type correctly:

[scrapy.core.scraper] Error downloading <POST https://xxx-splash.scrapinghub.com/render.json>
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
    result = g.send(result)
  File "/usr/local/lib/python2.7/site-packages/scrapy/core/downloader/middleware.py", line 53, in process_response
    spider=spider)
  File "/usr/local/lib/python2.7/site-packages/scrapy_splash/middleware.py", line 387, in process_response
    response = self._change_response_class(request, response)
  File "/usr/local/lib/python2.7/site-packages/scrapy_splash/middleware.py", line 402, in _change_response_class
    response = response.replace(cls=respcls, request=request)
  File "/usr/local/lib/python2.7/site-packages/scrapy/http/response/text.py", line 54, in replace
    return Response.replace(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/scrapy/http/response/__init__.py", line 81, in replace
    return cls(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/scrapy_splash/response.py", line 33, in __init__
    super(_SplashResponseMixin, self).__init__(url, *args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'encoding'

scrapinghub-stack-scrapy:1.4 is used, scrapy-splash version is 0.7.2

lopuhin avatar Apr 05 '19 14:04 lopuhin

I’m not sure how to fix this without a way to reproduce it :frowning_face:

Gallaecio avatar Nov 26 '19 11:11 Gallaecio