facebook-scraper
facebook-scraper copied to clipboard
Published
20 hours ago •
kevinzg
kevinzg
Empty comments_full with cookies="cookies.json" or cookies="from_browser"
I have not been able to get the comments by downloading cookies or using the browser. I also do not receive any error messages.
url = "https://www.facebook.com/elnuevodia/posts/pfbid02ow1g1rpibVjGzB6rK8mP44E5qmpdp3FuUtYLRsmR4Bzcwr4GDoAtHCUTmeJvqZ9pl"
post_df_full = pd.DataFrame(columns = [])
for post in get_posts(post_urls=[url], cookies="cookies.json", options={"comments": True, "reactions": True, "progress": True}):
post_entry = post
fb_post_df = pd.DataFrame.from_dict(post_entry, orient='index')
fb_post_df = fb_post_df.transpose()
post_df_full = post_df_full.append(fb_post_df)
print(post['post_id']+' get')
I'm unable to reproduce this issue. The code:
url = "https://www.facebook.com/elnuevodia/posts/pfbid02ow1g1rpibVjGzB6rK8mP44E5qmpdp3FuUtYLRsmR4Bzcwr4GDoAtHCUTmeJvqZ9pl"
for post in get_posts(post_urls=[url], cookies="cookies.json", options={"comments": True, "progress": True}):
print(post["comments"], len(post["comments_full"]))
outputs 469 167 for me. Perhaps try set your Facebook language to English? I would also recommend not setting "reactions": True if you don't need it, as it implies potentially a lot of extra requests when used in this combination (getting reactions for each comment)
Outputs 469 0 for me.
I attach my files to find out if I am doing something wrong.
REDACTED
Same thing when using json file:
[{
"Host raw": "https://.facebook.com/",
"Name raw": "datr",
"Path raw": "/",
"Content raw": "nHfcYdfriWs7Peh1Uunu01Bw",
"Expires": "10-01-2024 14:15:28",
"Expires raw": "1704910528",
"Send for": "Encrypted connections only",
"Send for raw": "true",
"HTTP only raw": "true",
"SameSite raw": "no_restriction",
"This domain only": "Valid for subdomains",
"This domain only raw": "false",
"Store raw": "firefox-container-6",
"First Party Domain": ""
},
{
"Host raw": "https://.facebook.com/",
"Name raw": "locale",
"Path raw": "/",
"Content raw": "en_US",
"Expires": "26-05-2022 17:52:26",
"Expires raw": "1653601946",
"Send for": "Encrypted connections only",
"Send for raw": "true",
"HTTP only raw": "false",
"SameSite raw": "no_restriction",
"This domain only": "Valid for subdomains",
"This domain only raw": "false",
"Store raw": "firefox-container-6",
"First Party Domain": ""
},
{
"Host raw": "https://.facebook.com/",
"Name raw": "sb",
"Path raw": "/",
"Content raw": "nHfcYaTyZghVATrR8OKjy8Uy",
"Expires": "21-05-2024 20:47:48",
"Expires raw": "1716338868",
"Send for": "Encrypted connections only",
"Send for raw": "true",
"HTTP only raw": "true",
"SameSite raw": "no_restriction",
"This domain only": "Valid for subdomains",
"This domain only raw": "false",
"Store raw": "firefox-container-6",
"First Party Domain": ""
},
{
"Host raw": "https://.facebook.com/",
"Name raw": "wd",
"Path raw": "/",
"Content raw": "1903x966",
"Expires": "01-06-2022 11:09:29",
"Expires raw": "1654096169",
"Send for": "Encrypted connections only",
"Send for raw": "true",
"HTTP only raw": "false",
"SameSite raw": "lax",
"This domain only": "Valid for subdomains",
"This domain only raw": "false",
"Store raw": "firefox-container-6",
"First Party Domain": ""
},
{
"Host raw": "https://.facebook.com/",
"Name raw": "c_user",
"Path raw": "/",
"Content raw": "REDACTED",
"Expires": "27-05-2023 12:40:06",
"Expires raw": "1685205606",
"Send for": "Encrypted connections only",
"Send for raw": "true",
"HTTP only raw": "false",
"SameSite raw": "no_restriction",
"This domain only": "Valid for subdomains",
"This domain only raw": "false",
"Store raw": "firefox-container-6",
"First Party Domain": ""
},
{
"Host raw": "https://.facebook.com/",
"Name raw": "xs",
"Path raw": "/",
"Content raw": "REDACTED",
"Expires": "27-05-2023 12:40:06",
"Expires raw": "1685205606",
"Send for": "Encrypted connections only",
"Send for raw": "true",
"HTTP only raw": "true",
"SameSite raw": "no_restriction",
"This domain only": "Valid for subdomains",
"This domain only raw": "false",
"Store raw": "firefox-container-6",
"First Party Domain": ""
},
{
"Host raw": "https://.facebook.com/",
"Name raw": "fr",
"Path raw": "/",
"Content raw": "0fJ5pPUP6buLl9oaG.AWUa13uBNifxoEjVLvDKYD-bL1Y.BikP7m.VP.AAA.0.0.BikQmB.AWVdNIZe798",
"Expires": "25-08-2022 13:25:21",
"Expires raw": "1661448321",
"Send for": "Encrypted connections only",
"Send for raw": "true",
"HTTP only raw": "true",
"SameSite raw": "no_restriction",
"This domain only": "Valid for subdomains",
"This domain only raw": "false",
"Store raw": "firefox-container-6",
"First Party Domain": ""
},
{
"Host raw": "https://.facebook.com/",
"Name raw": "dpr",
"Path raw": "/",
"Content raw": "1.3636363636363635",
"Expires": "03-06-2022 13:26:06",
"Expires raw": "1654277166",
"Send for": "Encrypted connections only",
"Send for raw": "true",
"HTTP only raw": "false",
"SameSite raw": "no_restriction",
"This domain only": "Valid for subdomains",
"This domain only raw": "false",
"Store raw": "firefox-container-6",
"First Party Domain": ""
},
{
"Host raw": "https://.facebook.com/",
"Name raw": "presence",
"Path raw": "/",
"Content raw": "C%7B%22t3%22%3A%5B%5D%2C%22utc3%22%3A1653672374045%2C%22v%22%3A1%7D",
"Expires": "At the end of the session",
"Expires raw": "0",
"Send for": "Encrypted connections only",
"Send for raw": "true",
"HTTP only raw": "false",
"SameSite raw": "no_restriction",
"This domain only": "Valid for subdomains",
"This domain only raw": "false",
"Store raw": "firefox-container-6",
"First Party Domain": ""
}]
It's probably not a good idea to upload your cookies to this public forum, as anyone who can see them then has access to your Facebook account. I've edited your comment to redact them. When using your cookies, I get the same result I got before, 469 162
I don't know how to work the troubleshooting because I don't get any error messages. These are the versions of my packages. I don't know what else you could recommend me to find a solution.
# Name Version Build Channel
aiodns 3.0.0 pypi_0 pypi
aiohttp 3.8.1 pypi_0 pypi
aiohttp-socks 0.7.1 pypi_0 pypi
aiosignal 1.2.0 pypi_0 pypi
altair 4.2.0 pypi_0 pypi
appdirs 1.4.4 pypi_0 pypi
argon2-cffi 21.3.0 pypi_0 pypi
argon2-cffi-bindings 21.2.0 pypi_0 pypi
asttokens 2.0.5 pyhd8ed1ab_0 conda-forge
async-timeout 4.0.2 pypi_0 pypi
attrs 21.4.0 pypi_0 pypi
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 py_2 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
beautifulsoup4 4.11.1 pypi_0 pypi
bleach 5.0.0 pypi_0 pypi
blinker 1.4 pypi_0 pypi
bpemb 0.3.3 pypi_0 pypi
brotli 1.0.9 h8ffe710_7 conda-forge
brotli-bin 1.0.9 h8ffe710_7 conda-forge
brotlipy 0.7.0 py39hb82d6ee_1004 conda-forge
browser-cookie3 0.13.0 pypi_0 pypi
bs4 0.0.1 pypi_0 pypi
bzip2 1.0.8 h8ffe710_4 conda-forge
ca-certificates 2021.10.8 h5b45459_0 conda-forge
cachetools 5.1.0 pypi_0 pypi
catalogue 2.0.6 py39hcbf5309_2 conda-forge
cchardet 2.1.7 pypi_0 pypi
certifi 2021.10.8 pypi_0 pypi
cffi 1.15.0 py39h0878f49_0 conda-forge
charset-normalizer 2.0.12 pyhd8ed1ab_0 conda-forge
click 8.0.4 pypi_0 pypi
cloudpickle 2.1.0 pypi_0 pypi
colorama 0.4.4 pyh9f0ad1d_0 conda-forge
conllu 4.4.2 pypi_0 pypi
cryptography 36.0.2 pypi_0 pypi
cssselect 1.1.0 pypi_0 pypi
cycler 0.11.0 pyhd8ed1ab_0 conda-forge
cymem 2.0.6 py39h415ef7b_3 conda-forge
cython-blis 0.7.7 py39h5d4886f_1 conda-forge
dataclasses 0.8 pyhc8e2a94_3 conda-forge
datasets 1.18.4 pypi_0 pypi
dateparser 1.1.1 pypi_0 pypi
debugpy 1.6.0 py39h415ef7b_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pypi_0 pypi
demjson3 3.0.5 pypi_0 pypi
deprecated 1.2.13 pypi_0 pypi
dill 0.3.4 pypi_0 pypi
elastic-transport 8.1.2 pypi_0 pypi
elasticsearch 8.1.2 pypi_0 pypi
emoji 1.7.0 pypi_0 pypi
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
es-core-news-sm 3.3.0 pypi_0 pypi
et_xmlfile 1.0.1 py_1001 conda-forge
executing 0.8.3 pyhd8ed1ab_0 conda-forge
facebook-scraper 0.2.55 pypi_0 pypi
fake-useragent 0.1.11 pypi_0 pypi
fastjsonschema 2.15.3 pypi_0 pypi
filelock 3.6.0 pypi_0 pypi
flair 0.11.3 pypi_0 pypi
fonttools 4.32.0 py39hb82d6ee_0 conda-forge
freetype 2.10.4 h546665d_1 conda-forge
frozenlist 1.3.0 pypi_0 pypi
fsspec 2022.3.0 pypi_0 pypi
ftfy 6.1.1 pypi_0 pypi
funcy 1.17 pyhd8ed1ab_0 conda-forge
future 0.18.2 py39hcbf5309_5 conda-forge
gdown 4.4.0 pypi_0 pypi
gensim 4.2.0 py39h415ef7b_0 conda-forge
geographiclib 1.52 pypi_0 pypi
geopy 2.2.0 pypi_0 pypi
gitdb 4.0.9 pypi_0 pypi
gitpython 3.1.27 pypi_0 pypi
googletransx 2.4.2 pypi_0 pypi
huggingface-hub 0.5.1 pypi_0 pypi
hyperopt 0.2.7 pypi_0 pypi
idna 3.3 pyhd8ed1ab_0 conda-forge
importlib-metadata 3.10.1 pypi_0 pypi
intel-openmp 2022.0.0 h57928b3_3663 conda-forge
ipykernel 6.13.0 py39h832f523_0 conda-forge
ipython 8.2.0 py39hcbf5309_0 conda-forge
ipython-genutils 0.2.0 pypi_0 pypi
ipywidgets 7.7.0 pypi_0 pypi
janome 0.4.2 pypi_0 pypi
jbig 2.1 h8d14728_2003 conda-forge
jedi 0.18.1 py39hcbf5309_1 conda-forge
jeepney 0.8.0 pypi_0 pypi
jinja2 3.1.2 pyhd8ed1ab_0 conda-forge
joblib 1.1.0 pyhd8ed1ab_0 conda-forge
jpeg 9e h8ffe710_1 conda-forge
jsonschema 4.5.1 pypi_0 pypi
jupyter_client 7.2.2 pyhd8ed1ab_1 conda-forge
jupyter_core 4.9.2 py39hcbf5309_0 conda-forge
jupyterlab-pygments 0.2.2 pypi_0 pypi
jupyterlab-widgets 1.1.0 pypi_0 pypi
keyring 23.5.0 pypi_0 pypi
kiwisolver 1.4.2 py39h2e07f2f_1 conda-forge
konoha 4.6.5 pypi_0 pypi
langcodes 3.3.0 pyhd8ed1ab_0 conda-forge
langdetect 1.0.9 pypi_0 pypi
lcms2 2.12 h2a16943_0 conda-forge
lerc 3.0 h0e60522_0 conda-forge
libblas 3.9.0 14_win64_mkl conda-forge
libbrotlicommon 1.0.9 h8ffe710_7 conda-forge
libbrotlidec 1.0.9 h8ffe710_7 conda-forge
libbrotlienc 1.0.9 h8ffe710_7 conda-forge
libcblas 3.9.0 14_win64_mkl conda-forge
libdeflate 1.10 h8ffe710_0 conda-forge
libffi 3.4.2 h8ffe710_5 conda-forge
liblapack 3.9.0 14_win64_mkl conda-forge
libpng 1.6.37 h1d00b33_2 conda-forge
libsodium 1.0.18 h8d14728_1 conda-forge
libtiff 4.3.0 hc4061b1_3 conda-forge
libwebp 1.2.2 h57928b3_0 conda-forge
libwebp-base 1.2.2 h8ffe710_1 conda-forge
libxcb 1.13 hcd874cb_1004 conda-forge
libzlib 1.2.11 h8ffe710_1014 conda-forge
lxml 4.8.0 pypi_0 pypi
lz4 4.0.0 pypi_0 pypi
lz4-c 1.9.3 h8ffe710_1 conda-forge
m2w64-gcc-libgfortran 5.3.0 6 conda-forge
m2w64-gcc-libs 5.3.0 7 conda-forge
m2w64-gcc-libs-core 5.3.0 7 conda-forge
m2w64-gmp 6.1.0 2 conda-forge
m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge
markupsafe 2.1.1 py39hb82d6ee_1 conda-forge
matplotlib-base 3.5.1 py39h581301d_0 conda-forge
matplotlib-inline 0.1.3 pyhd8ed1ab_0 conda-forge
mechanicalsoup 1.1.0 pypi_0 pypi
mistune 0.8.4 pypi_0 pypi
mkl 2022.0.0 h0e2418a_796 conda-forge
more-itertools 8.13.0 pypi_0 pypi
mpld3 0.3 pypi_0 pypi
msys2-conda-epoch 20160418 1 conda-forge
multidict 6.0.2 pypi_0 pypi
multiprocess 0.70.12.2 pypi_0 pypi
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
murmurhash 1.0.7 py39h415ef7b_0 conda-forge
nbclient 0.6.3 pypi_0 pypi
nbconvert 6.5.0 pypi_0 pypi
nbformat 5.4.0 pypi_0 pypi
nest-asyncio 1.5.5 pyhd8ed1ab_0 conda-forge
networkx 2.8.2 pypi_0 pypi
nltk 3.6.7 pyhd8ed1ab_0 conda-forge
notebook 6.4.11 pypi_0 pypi
numexpr 2.8.0 mkl_py39h01db045_2 conda-forge
numpy 1.22.3 py39h6331f09_2 conda-forge
openjpeg 2.4.0 hb211442_1 conda-forge
openpyxl 3.0.9 pyhd8ed1ab_0 conda-forge
openssl 3.0.3 h8ffe710_0 conda-forge
overrides 3.1.0 pypi_0 pypi
packaging 21.3 pyhd8ed1ab_0 conda-forge
pandas 1.4.2 py39h2e25243_1 conda-forge
pandocfilters 1.5.0 pypi_0 pypi
parse 1.19.0 pypi_0 pypi
parso 0.8.3 pyhd8ed1ab_0 conda-forge
pathy 0.6.1 pyhd8ed1ab_0 conda-forge
patsy 0.5.2 pyhd8ed1ab_0 conda-forge
pbkdf2 1.3 pypi_0 pypi
pickleshare 0.7.5 py_1003 conda-forge
pillow 9.1.0 py39ha53f419_2 conda-forge
pip 22.0.4 pyhd8ed1ab_0 conda-forge
pptree 3.1 pypi_0 pypi
preshed 3.0.6 py39h415ef7b_2 conda-forge
prometheus-client 0.14.1 pypi_0 pypi
prompt-toolkit 3.0.29 pyha770c72_0 conda-forge
protobuf 3.20.1 pypi_0 pypi
psutil 5.9.0 py39hb82d6ee_1 conda-forge
pthread-stubs 0.4 hcd874cb_1001 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
py4j 0.10.9.5 pypi_0 pypi
pyaes 1.6.1 pypi_0 pypi
pyarrow 7.0.0 pypi_0 pypi
pycares 4.1.2 pypi_0 pypi
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pycryptodome 3.14.1 pypi_0 pypi
pydantic 1.8.2 py39hb82d6ee_2 conda-forge
pydeck 0.7.1 pypi_0 pypi
pyee 8.2.2 pypi_0 pypi
pygments 2.11.2 pyhd8ed1ab_0 conda-forge
pyldavis 3.3.1 pyhd8ed1ab_0 conda-forge
pympler 1.0.1 pypi_0 pypi
pyopenssl 22.0.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.8 pyhd8ed1ab_0 conda-forge
pyppeteer 1.0.2 pypi_0 pypi
pyquery 1.4.3 pypi_0 pypi
pyrsistent 0.18.1 pypi_0 pypi
pysentimiento 0.3.2 pypi_0 pypi
pysocks 1.7.1 py39hcbf5309_5 conda-forge
python 3.9.12 hcf16a7b_1_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-socks 2.0.3 pypi_0 pypi
python_abi 3.9 2_cp39 conda-forge
pytz 2022.1 pyhd8ed1ab_0 conda-forge
pytz-deprecation-shim 0.1.0.post0 pypi_0 pypi
pywin32 303 py39hb82d6ee_0 conda-forge
pywin32-ctypes 0.2.0 pypi_0 pypi
pywinpty 2.0.5 pypi_0 pypi
pyyaml 6.0 pypi_0 pypi
pyzmq 22.3.0 py39he46f08e_2 conda-forge
regex 2022.3.2 pypi_0 pypi
requests 2.27.1 pyhd8ed1ab_0 conda-forge
requests-html 0.10.0 pypi_0 pypi
responses 0.18.0 pypi_0 pypi
sacremoses 0.0.49 pypi_0 pypi
schedule 1.1.0 pypi_0 pypi
scikit-learn 1.0.2 pypi_0 pypi
scipy 1.8.0 py39hc0c34ad_1 conda-forge
seaborn 0.11.2 hd8ed1ab_0 conda-forge
seaborn-base 0.11.2 pyhd8ed1ab_0 conda-forge
secretstorage 3.3.2 pypi_0 pypi
segtok 1.5.11 pypi_0 pypi
semver 2.13.0 pypi_0 pypi
send2trash 1.8.0 pypi_0 pypi
sentencepiece 0.1.95 pypi_0 pypi
setuptools 62.1.0 py39hcbf5309_0 conda-forge
shellingham 1.4.0 pyh44b312d_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sklearn 0.0 pypi_0 pypi
smart_open 5.2.1 pyhd8ed1ab_0 conda-forge
smmap 5.0.0 pypi_0 pypi
snscrape 0.4.3.20220107.dev45+ged3ea94 pypi_0 pypi
soupsieve 2.3.2.post1 pypi_0 pypi
spacy 3.3.0 py39hf03a35d_0 conda-forge
spacy-legacy 3.0.9 pyhd8ed1ab_0 conda-forge
spacy-loggers 1.0.2 pyhd8ed1ab_0 conda-forge
sqlite 3.38.2 h8ffe710_0 conda-forge
sqlitedict 2.0.0 pypi_0 pypi
srsly 2.4.3 py39h415ef7b_1 conda-forge
stack_data 0.2.0 pyhd8ed1ab_0 conda-forge
statsmodels 0.13.2 py39h5d4886f_0 conda-forge
streamlit 1.9.1 pypi_0 pypi
tabulate 0.8.9 pypi_0 pypi
tbb 2021.5.0 h2d74725_1 conda-forge
terminado 0.15.0 pypi_0 pypi
thinc 8.0.15 py39hefe7e4c_0 conda-forge
threadpoolctl 3.1.0 pypi_0 pypi
tinycss2 1.1.1 pypi_0 pypi
tk 8.6.12 h8ffe710_0 conda-forge
tokenizers 0.12.1 pypi_0 pypi
toml 0.10.2 pypi_0 pypi
toolz 0.11.2 pypi_0 pypi
torch 1.11.0 pypi_0 pypi
tornado 6.1 py39hb82d6ee_3 conda-forge
tqdm 4.64.0 pyhd8ed1ab_0 conda-forge
traitlets 5.1.1 pyhd8ed1ab_0 conda-forge
transformers 4.18.0 pypi_0 pypi
typer 0.4.1 pyhd8ed1ab_0 conda-forge
typing-extensions 4.2.0 hd8ed1ab_1 conda-forge
typing_extensions 4.2.0 pyha770c72_1 conda-forge
tzdata 2022.1 pypi_0 pypi
tzlocal 4.2 pypi_0 pypi
ucrt 10.0.20348.0 h57928b3_0 conda-forge
unicodedata2 14.0.0 py39hb82d6ee_1 conda-forge
urllib3 1.26.9 pyhd8ed1ab_0 conda-forge
validators 0.19.0 pypi_0 pypi
vc 14.2 hb210afc_6 conda-forge
vs2015_runtime 14.29.30037 h902a5da_6 conda-forge
w3lib 1.22.0 pypi_0 pypi
wasabi 0.9.1 pyhd8ed1ab_0 conda-forge
watchdog 2.1.8 pypi_0 pypi
wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge
webencodings 0.5.1 pypi_0 pypi
websockets 10.3 pypi_0 pypi
wheel 0.37.1 pyhd8ed1ab_0 conda-forge
widgetsnbextension 3.6.0 pypi_0 pypi
wikipedia-api 0.5.4 pypi_0 pypi
win_inet_pton 1.1.0 py39hcbf5309_4 conda-forge
wrapt 1.14.1 pypi_0 pypi
xorg-libxau 1.0.9 hcd874cb_0 conda-forge
xorg-libxdmcp 1.1.3 hcd874cb_0 conda-forge
xxhash 3.0.0 pypi_0 pypi
xz 5.2.5 h62dcd97_1 conda-forge
yarl 1.7.2 pypi_0 pypi
zeromq 4.3.4 h0e60522_1 conda-forge
zipp 3.8.0 pypi_0 pypi
zlib 1.2.11 h8ffe710_1014 conda-forge
zstd 1.5.2 h6255e5f_0 conda-forge
Try enable debug logging as per the issue template https://github.com/kevinzg/facebook-scraper/issues/273
I have the same issue: when I specify the cookies.txt file in get_posts, I can gather just the posts (no comments returned).
I used cookies.txt chrome extension to generate the cookies file.
Please anyone to explain us how to generate the right cookies.txt, and how to use it to expand the data to extract ?
when I click on cookies.txt I got: No cookies found for web.facebook.com How can I geenrate the for facebook?
@dihiaselma Could you post the code you're using? Try go to m.facebook.com instead of web.facebook.com