fbcrawl
fbcrawl copied to clipboard
rugantio
A Facebook crawler
Today i got this error, it was working fine until yesterday: ``` 2020-04-16 15:55:18 [facebook] INFO: Scraping facebook page https://mbasic.facebook.com/DonaldTrump 2020-04-16 15:55:20 [facebook] INFO: First page scraped, clicking on "more"!...
I am running fbcrawl from VMWare Windows 10 image. I have python 3.7, and I am seeing following error when I run the command: 2020-03-11 19:31:57 [fb] INFO: Going through...
Hello I found your project last night and installed it today. My primary interest lies with scraping comments. I ran the Trump comment crawl example which fails. After reading related...
I can't run the example command to crawl Donald Trump page because crawler redirect wrong url of page after login. This is log: ``` 2019-07-22 14:23:59 [fb] INFO: Email and...
For instructions on scraping comments I changed 'page' to 'post', which actually works at the moment. I know that 'page' should work. I will add another pr if I can...
ADDITIONALLY: On comments.py, there is a section of code that begins with "if back". This part is checking whether or not it needs to iterate upwards to get the rest...
Hi, I'm crawling the comments of one of famous guy's post. Some comments are up to 50000+ and I found that the scrapy engine would stop to crawler when it...
I tried to crawl a post with 2700 comments. But I can only run it to page 60 The post link: m.facebook.com/story.php?story_fbid=2226458920929531&id=2226454927596597&**`p=60`**&av=100036884506828&eav=AfaGbbgANEKjTi_nwspovtij7sx25oDoiBkQDA3hr_fqX5KDrdqrLBeclI6ydoKspu8&refid=52 The command: `scrapy crawl comments -a email="MAIL" -a...
[scrapy.spidermiddlewares.httperror] INFO: Ignoring response : HTTP status code is not handled or not allowed
Hi, When I make a test with Donald Trump I have this result, can you help me ? : INFO: Ignoring response : HTTP status code is not handled or...
