Can't scrape more than 29201 comments

[depocoder]

Hello, I'm trying to scrape 60k comments, but after 29k scraper finish his job. I tried to create script by my self, but it very alike this lib. Please help me. Video for testing.

last log

DEBUG:urllib3.connectionpool:https://www.youtube.com:443 "POST /youtubei/v1/next?key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8 HTTP/1.1" 200 None

[d0tN3t]

@depocoder

Hmm... I believe 443 is a timeout. This might solve your problem or guide you on what's happening. You may need to increase the timeout per the solution in the link.

https://stackoverflow.com/questions/57994570/readtimeout-httpsconnectionpoolhost-port-443-read-timed-out-read-timeo

[SKevo18]

@d0tN3t I believe 443 is a timeout

That is not the status code for timeout (408 is), it is the port for secured HTTPS traffic (https://www.lifewire.com/what-is-port-443-4690657). The real status code is actually 200 (OK), and you can see it at the end of the log file, before "None".

I believe that timeouts should either have status of 408 or raise ConnectionTimeoutError. I think that the issue is therefore located elsewhere

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 7 days with no activity.