FacebookPostsScraper icon indicating copy to clipboard operation
FacebookPostsScraper copied to clipboard

KeyError: 'story_fbid'

Open Rahulsunny11 opened this issue 4 years ago • 2 comments

Hey, I ran your scripts as python main.py and I am got error given below: File "main.py", line 25, in main data = fps.get_posts_from_list(profiles) File "C:\Users\ADMIN\Downloads\Compressed\FacebookPostsScraper-master\FacebookPostsScraper.py", line 133, in get_posts_from_list posts = self.get_posts_from_profile(profile) File "C:\Users\ADMIN\Downloads\Compressed\FacebookPostsScraper-master\FacebookPostsScraper.py", line 193, in get_posts_from_profile post_url = f'{p_url.scheme}://{p_url.hostname}{p_url.path}?story_fbid={qs["story_fbid"][0]}&id={qs["id"][0]}' KeyError: 'story_fbid' also can you tell f'{p_url.scheme}://{p_url.hostname}{p_url.path}?story_fbid={qs["story_fbid"][0]}&id={qs["id"][0]} where did this come from

It would great help if you are also including reactions, likes, haha, sad, comments, shares column

thank you

Rahulsunny11 avatar Jul 10 '20 06:07 Rahulsunny11

So, I experienced the same issue. Within the library's file (FacebookPostsScraper) when they are handling cleaning the post link (lines 185 to 197), they are only handling the case of the story_fbid. There are posts where an 'fbid' needs to be handled; in other words, there is no story_fbid - only an fbid.

I altered the script by adding a variable called id_key that captures the first key name returned from the parse_qs queryl and added another condition where you see the elif block. This handled the issue for me. Now, there may be others that are not handled, but for this scenario, it handled well:

# Clean the post link
            if post_url is not None:
                post_url = post_url.get('href', '')
                if len(post_url) > 0:
                    post_url = f'https://www.facebook.com{post_url}'
                    p_url = urlparse(post_url)
                    qs = parse_qs(p_url.query)
                    id_key = next(iter(qs))
                    if not is_group and id_key == 'story_fbid':
                        post_url = f'{p_url.scheme}://{p_url.hostname}{p_url.path}?story_fbid={qs["story_fbid"][0]}&id={qs["id"][0]}'
                    
                    elif not is_group and id_key == 'fbid':
                            post_url = f'{p_url.scheme}://{p_url.hostname}{p_url.path}?fbid={qs["fbid"][0]}&id={qs["id"][0]}'
                        
                    
                    else:
                        post_url = f'{p_url.scheme}://{p_url.hostname}{p_url.path}/permalink/{qs["id"][0]}/'
            else:
                post_url = ''

rsorma04 avatar Jul 15 '20 00:07 rsorma04

As of January 2020 @rsorma04 's solution worked for me! Thank you :D

NicoSerranoP avatar Jan 08 '21 02:01 NicoSerranoP