facebook-scraper icon indicating copy to clipboard operation
facebook-scraper copied to clipboard

Unable to extract all comments and replies of the facebook post

Open ajitkshirsagar opened this issue 2 years ago • 2 comments

While running the scrapper to extract post, comments, and reactions, I am getting the following exceptions:

 File ".pyenv/versions/3.9.1/lib/python3.9/site-packages/facebook_scraper/utils.py", line 279, in safe_consume
    for item in generator:
  File ".pyenv/versions/3.9.1/lib/python3.9/site-packages/facebook_scraper/extractors.py", line 1140, in extract_comment_replies
    for action in data['payload']['actions']:
TypeError: 'NoneType' object is not subscriptable
Exception when consuming <itertools.chain object at 0x109141f40>: <class 'TypeError'>: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/.pyenv/versions/3.9.1/lib/python3.9/site-packages/facebook_scraper/utils.py", line 279, in safe_consume
    for item in generator:
  File "/.pyenv/versions/3.9.1/lib/python3.9/site-packages/facebook_scraper/extractors.py", line 1140, in extract_comment_replies
    for action in data['payload']['actions']:
TypeError: 'NoneType' object is not subscriptable
Exception when consuming <itertools.chain object at 0x109176760>: <class 'TypeError'>: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/.pyenv/versions/3.9.1/lib/python3.9/site-packages/facebook_scraper/utils.py", line 279, in safe_consume
    for item in generator:
  File "/.pyenv/versions/3.9.1/lib/python3.9/site-packages/facebook_scraper/extractors.py", line 1140, in extract_comment_replies
    for action in data['payload']['actions']:
TypeError: 'NoneType' object is not subscriptable

I am running the scrapper using the following:

facebook-scraper --filename fb-data-part0.json --timeout 10000 --comments --reactions --reactors --format=json --days-limit 70 --verbose --cookies cookies.txt --resume-file resume.txt anwb

ajitkshirsagar avatar Sep 02 '22 08:09 ajitkshirsagar

@ajitkshirsagar did you solve it?

pribadihcr avatar Sep 27 '23 09:09 pribadihcr

@ajitkshirsagar did you solve it?

@pribadihcr I managed to scrap some data with an actual user log-in session but after some time Facebook also found this activity and kicked me out.

ajitkshirsagar avatar Oct 11 '23 07:10 ajitkshirsagar