Raki Lachraf

icon location El Oued, Algeria icon location I am passionate about new professional experiences and my key skill is learning on the go, which is what the human brain is made for after all.

Results 3 comments of Raki Lachraf

Cannot crawl more than 60 pages comment

I'm facing a similar a problem but not just 60 comments I'm getting just 1.8k out of 165k comments ! did you acquire any solutions ?

Cannot crawl more than 60 pages comment

DUPEFILTER_CLASS = 'scrapy.dupefilters.BaseDupeFilter' try adding this line to settings.py file of fbcrawl project according to the documentation, scrapy's spider calls off when encountering duplicate links (exactly what usually happens with...

csv file limit

yes and you can change it: go to line 75 in fbcrawl.py , increase 10e5 to 10e6 (a million) or whatever you want...