youtube-dl
youtube-dl copied to clipboard
ytdl-org
HTTP Error 503: Service Unavailable on www3.nhk.or.jp
Checklist
- [*] I'm reporting a broken site support
- [*] I've verified that I'm running youtube-dl version 2021.12.17
- [*] I've checked that all provided URLs are alive and playable in a browser
- [*] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [*] I've searched the bugtracker for similar issues including closed ones
Verbose log
$ ./youtube-dl -v https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2032293/
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2032293/']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2023.08.07 [86e3cf5e5] (single file build)
[debug] ** This version was built from the latest master code at https://github.com/ytdl-org/youtube-dl.
[debug] ** For support, visit the main site.
[debug] Python 3.11.4 (CPython x86_64 64bit) - Linux-6.4.2-060402-generic-x86_64-with-glibc2.37 - OpenSSL 3.0.8 7 Feb 2023 - glibc 2.37
[debug] exe versions: ffmpeg 5.1.2, ffprobe 5.1.2, rtmpdump 2.4
[debug] Proxy map: {}
[NhkVod] 2032-293: Downloading JSON metadata
[Piksel] nw_vod_v_en_2032_293_20230803113000_01_1691031888: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 503: Service Unavailable (caused by <HTTPError 503: 'Service Unavailable'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "/storage/tmp/./youtube-dl/youtube_dl/extractor/common.py", line 665, in _request_webpage
return self._downloader.urlopen(url_or_request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/tmp/./youtube-dl/youtube_dl/YoutubeDL.py", line 2465, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 525, in open
response = meth(req, response)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 634, in http_response
response = self.parent.error(
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 563, in error
return self._call_chain(*args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "/usr/lib/python3.11/urllib/request.py", line 643, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
Description
For some time, youtube-dl no longer works on NHK World, i.e. https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2032293/
It errors as above. Tried the last "stable" version 2021.12.17 and the last "nightly" (2023.08.07).
Looks like we need to merge https://github.com/ytdl-org/youtube-dl/pull/32213. By all means check out the PR if you wish.
we need to merge #32213. By all means, check out the PR
Woks basically OK, at least with the video in the OP:
yt-dl -vF "https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2032293/" =>
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ffmpeg-location', '.\\FFmpeg', '--external-downloader-args', '-v 8 -stats', '-vF', 'https://www3.nhk.or.jp/nhkworld/en/ondemand/video/2032293/']
[debug] Encodings: locale cp1253, fs mbcs, out cp737, pref cp1253
[debug] youtube-dl version 2023.08.04.2154
[debug] Lazy loading extractors enabled
[debug] Single file build
[debug] Python 3.4.10 (CPython x86 32bit) - Windows-Vista-6.0.6003-SP2 - OpenSSL 1.0.2k 26 Jan 2017
[debug] exe versions: ffmpeg n6.1-dev-1700-N-111584-ga4e6168, ffprobe n6.1-dev-1700-N-111584-ga4e6168, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[NhkVod] 2032-293: Downloading JSON metadata
[Piksel] nw_vod_v_en_2032_293_20230803113000_01_1691031888: Downloading webpage
[Piksel] nw_vod_v_en_2032_293_20230803113000_01_1691031888: Downloading JSON metadata
[Piksel] nw_vod_v_en_2032_293_20230803113000_01_1691031888: Downloading JSON metadata
[Piksel] m2k9n530: Downloading m3u8 information
[Piksel] m2k9n530: Downloading SMIL file
[info] Available formats for m2k9n530:
format code extension resolution note
http-383 mp4 320x180 383k video@ 256k, audio@127k, 77.40MiB
hls-392 mp4 unknown 392k
http-392 mp4 320x? 392k
http-663-0 mp4 640x360 663k video@ 536k, audio@127k, 133.55MiB
http-663-1 mp4 640x360 663k video@ 536k, audio@127k, 133.55MiB
hls-678 mp4 unknown 678k
http-678 mp4 640x? 678k
http-1242 mp4 1280x720 1242k video@1115k, audio@127k, 249.57MiB
hls-1271 mp4 unknown 1271k
http-1271 mp4 1280x? 1271k
http-2126 mp4 1280x720 2126k video@1999k, audio@127k, 426.57MiB
http-2177 mp4 1280x? 2177k
http-10268 mp4 1920x1080 10268k video@10268k, audio@ 0k, 2.06GiB (best)
... Some of the metadata details in the resolution and note columns are still missing (unknown, ?) or appear incorrect (e.g., is the best format really video-only? video@10268k, audio@ 0k suggests that 😉 ) ...
... To be fair 😜 , "downstream" aren't doing any better in the metadata department, so I guess it's because of inherent limitations in the NHKWorld+Piksel IEs:
[debug] Sort order given by extractor: tbr
[debug] Formats sorted by: hasvid, ie_pref, tbr, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for m2k9n530:
ID EXT RESOLUTION | FILESIZE TBR PROTO | VCODEC VBR ACODEC ABR
---------------------------------------------------------------------------------
http-383 mp4 320x180 | 77.40MiB 383k https | unknown 256k unknown 127k
hls-392 mp4 unknown | 392k m3u8 | unknown unknown
http-392 mp4 320x? | 392k https | unknown unknown
http-663-0 mp4 640x360 | 133.55MiB 663k https | unknown 536k unknown 127k
http-663-1 mp4 640x360 | 133.55MiB 663k https | unknown 536k unknown 127k
hls-678 mp4 unknown | 679k m3u8 | unknown unknown
http-678 mp4 640x? | 679k https | unknown unknown
http-1242 mp4 1280x720 | 249.57MiB 1242k https | unknown 1115k unknown 127k
hls-1271 mp4 unknown | 1272k m3u8 | unknown unknown
http-1271 mp4 1280x? | 1272k https | unknown unknown
http-2126 mp4 1280x720 | 426.57MiB 2126k https | unknown 1999k unknown 127k
http-2177 mp4 1280x? | 2177k https | unknown unknown
http-10268 mp4 1920x1080 | 2.06GiB 10268k https | unknown 10268k unknown
Looks like we need to merge https://github.com/ytdl-org/youtube-dl/pull/32213. By all means check out the PR if you wish.
Is there any reason to not merge that PR?
Comments with successful or other experiences there would incentivise the process. Otherwise it has to wait until I have time to exercise it myself.