funda-scraper icon indicating copy to clipboard operation
funda-scraper copied to clipboard

AttributeError: 'FundaSpider' object has no attribute '_rules'

Open jeroenvermunt opened this issue 3 years ago • 1 comments

When running the code after forking, I immediately got the following error:

AttributeError: 'FundaSpider' object has no attribute '_rules'

I solved it by adding the following line to init():

super(FundaSoldSpider, self).__init__(place=place)

The error stopped appearing, but now the following lines show up during scraping, giving an empty json file in the end.

[scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.funda.nl/koop/verkocht/eindhoven/p259/> (referer: None)
[scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (301) to <GET https://www.funda.nl/koop/verkocht/eindhoven/p268/> from <GET http://www.funda.nl/koop/verkocht/eindhoven/p268/>

there are the following possibilities:

  1. the fix solved the error but broke the scraper
  2. the fix solved the error, but the scraper does not work on a different aspect.

I'm am not sure which one it is, anyone have any idea on how to get the scraper to work?

jeroenvermunt avatar Dec 29 '21 21:12 jeroenvermunt

This fixes those problems + some others that prevent it from working, just in case you didn't figure it out yet! (only changes for the currently for-sale houses, not the sold ones)

haydnjm avatar Mar 12 '22 18:03 haydnjm