you-get
you-get copied to clipboard
soimort
ixigua video download failed 西瓜视频下载视频
you-get https://www.ixigua.com/i6631065141750268420 --debug
output:
you-get: version 0.4.1488, a tiny downloader that scrapes the web.
you-get: Namespace(URL=['https://www.ixigua.com/6851090298882392584'], auto_rename=False, cookies=None, debug=True, extractor_proxy=None, force=False, format=None, help=False, http_proxy=None, info=False, input_file=None, insecure=False, itag=None, json=False, no_caption=False, no_merge=False, no_proxy=False, output_dir='.', output_filename=None, password=None, player=None, playlist=False, skip_existing_file_size_check=False, socks_proxy=None, stream=None, timeout=600, url=False, version=False)
Traceback (most recent call last):
File "/Users/xx/project/py37/bin/you-get", line 8, in <module>
sys.exit(main())
File "/Users/xx/project/py37/lib/python3.7/site-packages/you_get/__main__.py", line 92, in main
main(**kwargs)
File "/Users/xx/project/py37/lib/python3.7/site-packages/you_get/common.py", line 1799, in main
script_main(any_download, any_download_playlist, **kwargs)
File "/Users/xx/project/py37/lib/python3.7/site-packages/you_get/common.py", line 1687, in script_main
**extra
File "/Users/xx/project/py37/lib/python3.7/site-packages/you_get/common.py", line 1328, in download_main
download(url, **kwargs)
File "/Users/xx/project/py37/lib/python3.7/site-packages/you_get/common.py", line 1790, in any_download
m.download(url, **kwargs)
File "/Users/xx/project/py37/lib/python3.7/site-packages/you_get/extractors/ixigua.py", line 93, in ixigua_download
conf = loads(match1(html, r"window\.config = (.+);"))
File "/Users/xx/.pyenv/versions/3.7.9/lib/python3.7/json/__init__.py", line 341, in loads
raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType
Hello @gojuukaze, Thanks for the Pull Request. We :heart: our contributors! Please wait for one of our human maintainers to review your patches. This may take a few days to weeks. Also, please understand that although your Pull Request may or may not be eventually merged, we value all contributions equally.
祝您健康!
you-get --debug https://github.com/soimort/you-get/wiki/Known-Bugs
输出：
[DEBUG] get_location: https://github.com/soimort/you-get/wiki/Known-Bugs [DEBUG] get_location: https://github.com/soimort/you-get/wiki/Known-Bugs you-get: version 0.4.1527, a tiny downloader that scrapes the web. you-get: Namespace(version=False, help=False, info=False, url=False, json=False, no_merge=False, no_caption=False, force=False, skip_existing_file_size_check=False, format=None, output_filename=None, output_dir='.', player=None, cookies=None, timeout=600, debug=True, input_file=None, password=None, playlist=False, first=None, last=None, size=None, auto_rename=False, insecure=False, http_proxy=None, extractor_proxy=None, no_proxy=False, socks_proxy=None, stream=None, itag=None, URL=['https://github.com/soimort/you-get/wiki/Known-Bugs']) Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1346, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1257, in request self._send_request(method, url, body, headers, encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1303, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1252, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1012, in _send_output self.send(msg) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 952, in send self.connect() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1426, in connect self.sock = self._context.wrap_socket(self.sock, File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 500, in wrap_socket return self.sslsocket_class._create( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1040, in _create self.do_handshake() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1309, in do_handshake self._sslobj.do_handshake() ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/you_get/common.py", line 1792, in url_to_module location = get_location(url) # t.co isn't happy with fake_headers File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/you_get/common.py", line 393, in get_location res = urlopen_with_retry(req) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/you_get/common.py", line 408, in urlopen_with_retry return request.urlopen(*args, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 214, in urlopen return opener.open(url, data, timeout) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 517, in open response = self._open(req, data) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 534, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain result = func(*args) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1389, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1349, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)>
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1346, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1257, in request self._send_request(method, url, body, headers, encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1303, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1252, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1012, in _send_output self.send(msg) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 952, in send self.connect() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1426, in connect self.sock = self._context.wrap_socket(self.sock, File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 500, in wrap_socket return self.sslsocket_class._create( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1040, in _create self.do_handshake() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1309, in do_handshake self._sslobj.do_handshake() ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/bin/you-get", line 8, in
同样的情况，MAC 系统 Python 3.9.6
you-get -i --debug https://www.iq.com/play/%E4%BD%A0%E7%9A%84%E5%90%8D%E5%AD%97%E6%88%91%E7%9A%84%E5%A7%93%E6%B0%8F-17aclssabth?frmrp=home&frmb=focus_banner_1&frmrs=click
[DEBUG] get_location: https://www.iq.com/play/%E4%BD%A0%E7%9A%84%E5%90%8D%E5%AD%97%E6%88%91%E7%9A%84%E5%A7%93%E6%B0%8F-17aclssabth?frmrp=home you-get: version 0.4.1536, a tiny downloader that scrapes the web. you-get: Namespace(version=False, help=False, info=True, url=False, json=False, no_merge=False, no_caption=False, force=False, skip_existing_file_size_check=False, format=None, output_filename=None, output_dir='.', player=None, cookies=None, timeout=600, debug=True, input_file=None, password=None, playlist=False, first=None, last=None, size=None, auto_rename=False, insecure=False, http_proxy=None, extractor_proxy=None, no_proxy=False, socks_proxy=None, stream=None, itag=None, URL=['https://www.iq.com/play/%E4%BD%A0%E7%9A%84%E5%90%8D%E5%AD%97%E6%88%91%E7%9A%84%E5%A7%93%E6%B0%8F-17aclssabth?frmrp=home']) Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1346, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1257, in request self._send_request(method, url, body, headers, encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1303, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1252, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1012, in _send_output self.send(msg) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 952, in send self.connect() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1426, in connect self.sock = self._context.wrap_socket(self.sock, File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 500, in wrap_socket return self.sslsocket_class._create( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1040, in _create self.do_handshake() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1309, in do_handshake self._sslobj.do_handshake() ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/you_get/common.py", line 1810, in url_to_module location = get_location(url) # t.co isn't happy with fake_headers File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/you_get/common.py", line 393, in get_location res = urlopen_with_retry(req) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/you_get/common.py", line 408, in urlopen_with_retry return request.urlopen(*args, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 214, in urlopen return opener.open(url, data, timeout) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 517, in open response = self._open(req, data) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 534, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain result = func(*args) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1389, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1349, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)>
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1346, in do_open h.request(req.get_method(), req.selector, req.data, headers, File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1257, in request self._send_request(method, url, body, headers, encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1303, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1252, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1012, in _send_output self.send(msg) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 952, in send self.connect() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1426, in connect self.sock = self._context.wrap_socket(self.sock, File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 500, in wrap_socket return self.sslsocket_class._create( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1040, in _create self.do_handshake() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1309, in do_handshake self._sslobj.do_handshake() ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/bin/you-get", line 8, in
貌似现在都还没解决
you-get https://www.ixigua.com/6993139705197888031 --debug
you-get: version 0.4.1536, a tiny downloader that scrapes the web.
you-get: Namespace(URL=['https://www.ixigua.com/6993139705197888031'], auto_rename=False, cookies=None, debug=True, extractor_proxy=None, first=None, force=False, format=None, help=False, http_proxy=None, info=False, input_file=None, insecure=False, itag=None, json=False, last=None, no_caption=False, no_merge=False, no_proxy=False, output_dir='.', output_filename=None, password=None, player=None, playlist=False, size=None, skip_existing_file_size_check=False, socks_proxy=None, stream=None, timeout=600, url=False, version=False)
Traceback (most recent call last):
File "/Users/twan/Library/Python/3.8/bin/you-get", line 8, in
我初步检查了一下问题。可能是新的反爬或者网页改版，现在获取的内容和以往不同。以下经验留给大家作为参考。
首先是原先 85 行 resp = urlopen_with_retry(request.Request(url)) 的获取的内容和以往不同，导致后面的 resp.getheader('Set-Cookie') 返回值为 None，进而报错。我通过以下代码添加了一个 header 来改进：
ua = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36'
headers = {'Accept': '*/*', 'Accept-Language': 'en-US,en;q=0.5', 'User-Agent': ua}
resp = urlopen_with_retry(request.Request(url, headers=headers))
返回头中包含了后面需要的 Set-Cookie，但是 HTML 内容依然有误，不包含 window.config 相关内容。由于时间有限，没能完整阅读和理解全部代码并解决问题，留给其他人或者我有空时解决（暂时没空了）。
我初步检查了一下问题。可能是新的反爬或者网页改版，现在获取的内容和以往不同。以下经验留给大家作为参考。
首先是原先 85 行
resp = urlopen_with_retry(request.Request(url))的获取的内容和以往不同，导致后面的resp.getheader('Set-Cookie')返回值为None，进而报错。我通过以下代码添加了一个 header 来改进：ua = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36' headers = {'Accept': '*/*', 'Accept-Language': 'en-US,en;q=0.5', 'User-Agent': ua} resp = urlopen_with_retry(request.Request(url, headers=headers))返回头中包含了后面需要的
Set-Cookie，但是 HTML 内容依然有误，不包含window.config相关内容。由于时间有限，没能完整阅读和理解全部代码并解决问题，留给其他人或者我有空时解决（至少一个月后了）。
今天我搜索这个问题的时候发现还没有修复，然后我去下载了他的分支代码，发现他只提交了一个test文件的几行代码，所以我做了一个修复，合并请求在这里，请你有空的时候看一下。
