andi icon indicating copy to clipboard operation
andi copied to clipboard

andi does not support functools.partial

Open victor-torres opened this issue 4 years ago • 0 comments

I'm trying to use scrapy-po – which relies on andi – with a callback provided by functools.partial but I'm facing the following traceback.

Andi is internally trying to use typing.get_type_hints to inspect the target element and build the andi.Plan while this built-in method does not accept anything that "is not a module, class, method, or function".

Traceback (most recent call last):
  File "/Users/victortorres/.virtualenvs/scrapy-po/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/Users/victortorres/.virtualenvs/scrapy-po/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/Users/victortorres/.virtualenvs/scrapy-po/lib/python3.7/site-packages/scrapy/core/downloader/middleware.py", line 51, in process_response
    response = yield deferred_from_coro(method(request=request, response=response, spider=spider))
  File "/Users/victortorres/.virtualenvs/scrapy-po/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/Users/victortorres/git/shub/scrapy-po/scrapy_po/middleware.py", line 30, in process_response
    plan, provider_instances = build_plan(callback, response)
  File "/Users/victortorres/git/shub/scrapy-po/scrapy_po/middleware.py", line 53, in build_plan
    externally_provided=provider_instances.keys()
  File "/Users/victortorres/.virtualenvs/scrapy-po/lib/python3.7/site-packages/andi/andi.py", line 264, in plan
    dependency_stack=None)
  File "/Users/victortorres/.virtualenvs/scrapy-po/lib/python3.7/site-packages/andi/andi.py", line 294, in _plan
    arguments = inspect(class_or_func)
  File "/Users/victortorres/.virtualenvs/scrapy-po/lib/python3.7/site-packages/andi/andi.py", line 24, in inspect
    annotations = get_type_hints(func, globalns)
  File "/Users/victortorres/miniconda3/lib/python3.7/typing.py", line 993, in get_type_hints
    'or function.'.format(obj))
TypeError: functools.partial(<bound method PartialCallbackSpider.parse of <PartialCallbackSpider 'test_spider' at 0x10f487dd8>>, use_x=True) is not a module, class, method, or function.

Example spider:

class MySpider(scrapy.Spider):

    def start_requests(self):
        yield scrapy.Request('http://example.com/', functools.partial(self.parse, use_x=True))

    def parser(self, response, use_x):
        return {
            'url': response.url,
            'use_x': use_x,
        }

I've had discussed this behavior with @kmike and he has asked me to report this bug here.

victor-torres avatar Apr 13 '20 20:04 victor-torres