pythoncode-tutorials icon indicating copy to clipboard operation
pythoncode-tutorials copied to clipboard

The Youtube comment scraper seems to be down.

Open Hypurl opened this issue 4 years ago • 3 comments

Getting error: `runfile('C:/Users/Jeff/Downloads/untitled1.py', wdir='C:/Users/Jeff/Downloads') Traceback (most recent call last):

File "C:\Users\Jeff\Downloads\untitled1.py", line 120, in for count, comment in enumerate(get_comments(url)):

File "C:\Users\Jeff\Downloads\untitled1.py", line 50, in get_comments data = json.loads(data_str)

File "C:\Users\Jeff\anaconda3\lib\json_init_.py", line 348, in loads return _default_decoder.decode(s)

File "C:\Users\Jeff\anaconda3\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())

File "C:\Users\Jeff\anaconda3\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None

JSONDecodeError: Expecting value`

Hypurl avatar Dec 30 '20 11:12 Hypurl

It's not working for me either. I'm getting the same JSONDecodeError/Traceback that you are.

sarahelizabeth avatar Apr 19 '21 17:04 sarahelizabeth

I'm having the same issue too, while I'll try to fix the code, please refer to YouTube API tutorial instead (and that's what I'm currently using as well):

https://www.thepythoncode.com/article/using-youtube-api-in-python

Hope this helps!

x4nth055 avatar Apr 25 '21 01:04 x4nth055

I understand this is an old issue, but I have found a fix for those who are still expecting one.

YouTube now uses var ytInitialData instead of window["ytInitialData"] . You can change line 50, or any line close by that creates the data_str variable from this:

data_str = find_value(
    res.text, 'window["ytInitialData"] = ', num_sep_chars=0, separator="\n"
).rstrip(";")

to this:

data_str = find_value(
    res.text, 'var ytInitialData = ', num_sep_chars=0, separator="\n"
).rstrip(";").split(';</script>')[0]

This script scrapes less comments than it should, although I don't know if it was always like that or not. Hope this helps!

ajskateboarder avatar Aug 05 '22 12:08 ajskateboarder