Facebook-event-crawler icon indicating copy to clipboard operation
Facebook-event-crawler copied to clipboard

Parse descriptions from HTML commented out

Open elvispoz opened this issue 5 years ago • 14 comments

Crawler does not download event descriptions...

elvispoz avatar May 18 '20 12:05 elvispoz

You may only need one xpath correction in https://github.com/DaWe35/Facebook-event-crawler/blob/cac0845468352423c061d943fbf334ed2651eb73/Crawlr.py#L260

DaWe35 avatar May 20 '20 16:05 DaWe35

I know, but what is correct ?

elvispoz avatar May 21 '20 18:05 elvispoz

Im testing with new selector but still does not effects...

event_description = tree.xpath('//div[@id="unit_id_886302548152152"]/section[1]/text()')

Have you got any idea?

elvispoz avatar May 22 '20 11:05 elvispoz

section[1]

That's weird, section[1] need to work. Anyway, I think there is a Facebook release slipping - in my old facebook account <div> works, in my new account <section>, so we need to support all of them. Cool....

Can you try out event_description = tree.xpath('//div[@id="unit_id_886302548152152"]/section/text()') ? Anyway I don't understand why section[1] not works...

DaWe35 avatar May 22 '20 13:05 DaWe35

Descriptions in mysql are still empty :(

elvispoz avatar May 25 '20 07:05 elvispoz

Have you got any idea and solution... ?

elvispoz avatar May 27 '20 10:05 elvispoz

Have you got any idea and solution... ?

Sorry @elvispoz, today I have no time, I'll check it out later. Sometimes I need to work for money also :)

DaWe35 avatar May 29 '20 12:05 DaWe35

@elvispoz I just registered a new Facebook account, and everything works fine. Can you give me your Facebook account to have a try? If you followed my guide, you registered one only for FB crawler, so there is no personal data. You can find me here: https://discord.gg/69SZC4v (I'm DaWe)

DaWe35 avatar May 31 '20 12:05 DaWe35

Hi, Have you got any time for this? I see thet fb make some change in code and put description to: <!-- xxx -->

elvispoz avatar Jul 13 '20 12:07 elvispoz

Maybe it will be some solution... https://stackoverflow.com/questions/44506990/getting-encoded-text-while-scraping-the-data-from-url-using-beautifulsoup-python

elvispoz avatar Jul 15 '20 08:07 elvispoz

Is there anyone having the same issue?

DaWe35 avatar Jul 29 '20 09:07 DaWe35

Is there anyone having the same issue?

Hello, I followed all your guidances, and the crawler worked smoothly. But there are no records downloaded, the 'events' table is always empty... I am not sure which part went wrong...

XinminHu avatar Jan 07 '21 12:01 XinminHu

Same here, the crawler is working, but 'events" table still empty...

oblab avatar Nov 12 '21 09:11 oblab

python3 Crawlr.py                                                                                                                                                         
Logging in to facebook...                                                                                                                                                                     
0 old row deleted                                                                                                                                                                             
Pages are already updated less than an hour ago, no new events queried  

oblab avatar Nov 12 '21 09:11 oblab