yt-dlp
yt-dlp copied to clipboard
yt-dlp
BBC iPlayer broken
DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
- [X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field
Checklist
- [X] I'm reporting that a supported site is broken
- [X] I've verified that I'm running yt-dlp version 2023.03.04 (update instructions) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
- [X] I've read the guidelines for opening an issue
- [X] I've read about sharing account credentials and I'm willing to share it if required
Region
Belgium (via UK VPN)
Provide a description that is worded well enough to be understood
It seems BBC iPlayer has changed streaming formats... making downloads from yt-dlp no longer working. The programmes can still be watched online and download fine using another program (+ using a VPN). E.g. : yt-dlp -vU -F https://www.bbc.co.uk/iplayer/episode/p0f21jy9/wild-isles-series-1-2-woodland
Provide verbose output that clearly demonstrates the problem
- [X] Run your yt-dlp command with -vU flag added (
yt-dlp -vU <your command line>) - [ ] If using API, add
'verbose': TruetoYoutubeDLparams instead - [X] Copy the WHOLE output (starting with
[debug] Command-line config) and insert it below
Complete Verbose Output
[debug] Command-line config: ['-vU', '-F', 'https://www.bbc.co.uk/iplayer/episode/p0f21jy9/wild-isles-series-1-2-woodland']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version [email protected] [392389b7d] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19044-SP0 (OpenSSL 1.1.1k 25 Mar 2021)
[debug] exe versions: ffmpeg n4.3.1-19-gaf2a430bb1, ffprobe n4.3.1-19-gaf2a430bb1
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1786 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: [email protected], Current version: [email protected]
Current Build Hash: 5590c57bd0433ed239a2deaaf92e2ad6f37fe50f53664c821575cafe106a9421
yt-dlp is up to date ([email protected])
[bbc.co.uk] Extracting URL: https://www.bbc.co.uk/iplayer/episode/p0f21jy9/wild-isles-series-1-2-woodland
[bbc.co.uk] p0f21jy9: Downloading video page
[bbc.co.uk] p0f21jy9: Downloading playlist JSON
[bbc.co.uk] p0fjrswh: Downloading media selection JSON
[bbc.co.uk] p0fjrswh: Downloading m3u8 information
[bbc.co.uk] p0fjrswh: Downloading m3u8 information
[bbc.co.uk] p0fjrswh: Downloading m3u8 information
[bbc.co.uk] p0fjrswh: Downloading m3u8 information
[bbc.co.uk] p0fjrswh: Downloading m3u8 information
[bbc.co.uk] p0fjrswh: Downloading m3u8 information
[bbc.co.uk] p0fjrswh: Downloading MPD manifest
[bbc.co.uk] p0fjrswh: Downloading MPD manifest
[bbc.co.uk] p0fjrswh: Downloading MPD manifest
[bbc.co.uk] p0fjrswh: Downloading MPD manifest
[bbc.co.uk] p0fjrswh: Downloading MPD manifest
[bbc.co.uk] p0fjrswh: Downloading MPD manifest
[bbc.co.uk] p0f21k7x: Downloading media selection JSON
[bbc.co.uk] p0f21k7x: Downloading media selection JSON
ERROR: [bbc.co.uk] p0f21jy9: bbc.co.uk returned error: selectionunavailable
File "yt_dlp\extractor\common.py", line 694, in extract
File "yt_dlp\extractor\bbc.py", line 576, in _real_extract
File "yt_dlp\extractor\bbc.py", line 463, in _download_playlist
File "yt_dlp\extractor\bbc.py", line 342, in _download_media_selector
File "yt_dlp\extractor\bbc.py", line 327, in _raise_extractor_error
Reproduced from UK with yt-dlp [email protected].
yt-dl release and master are working.
The error comes at l.342 when _download_media_selector_url() has returned a selectionunavailable error and no subsequent call has succeeded (which would have returned from the method).
This looks like the reason for the different behaviour
--- youtube_dl/extractor/bbc.py
+++ yt_dlp/extractor/bbc.py
...
@@ -461,9 +447,10 @@
playlist = self._download_json(
'http://www.bbc.co.uk/programmes/%s/playlist.json' % playlist_id,
playlist_id, 'Downloading playlist JSON')
-
- version = playlist.get('defaultAvailableVersion')
- if version:
+ formats = []
+ subtitles = {}
+
+ for version in playlist.get('allAvailableVersions', []):
smp_config = version['smpConfig']
title = smp_config['title']
description = smp_config['summary']
...
As this code is trying for multiple versions, any error should be masked until it can be determined whether the error needs to be propagated, similar to the logic in _download_media_selector_url().
This is a rough patch:
- version_formats, version_subtitles = self._download_media_selector(programme_id)
+ try:
+ version_formats, version_subtitles = self._download_media_selector(programme_id)
+ except ExtractorError as e:
+ self.report_warning(f'Unable to get version {version}: {e}')
+ continue
This only applied to Signed version of the programme, 9 out 30 videos I downloaded threw ERROR: local variable 'programme_id' referenced before assignment, such as this episode.
programme_id will be undefined if allAvailableVersions and defaultAvailableVersion are empty or null. For example, response from https://www.bbc.co.uk/programmes/b09ylv6t/playlist.json:
{
"info": {
"readme": "For the use of Radio, Music and Programmes only"
},
"statsObject": {
"parentPID": "b09ylv6t",
"parentPIDType": "episode"
},
"defaultAvailableVersion": null,
"allAvailableVersions": [],
"holdingImage": "//ichef.bbci.co.uk/images/ic/976x549/p062qjzg.jpg"
}
@dirkf Yes, now I have to use playwright to scrape the page, accept the cookie settings, trigger video play action, and click the subtitle menu(for the first video only, and some pages have no subtitle even for signed programmes):
dash-mpd-cli --write-subs --quality best 'https://mm.bidi.bbc.co.uk/vod-dash-uk/usp/auth/vod/piff_abr_full_hd/0e9fd7-m001dhbj/vf_m001dhbj_c45c4c9c-d225-49b3-9c62-693db5e4a179.ism/pc_hd_abr_v2_dash_master.mpd?at=xxx' --output "825. Emeli Sande - Too Many Tickles.mp4"
curl 'https://vod-sub-uk-live.akamaized.net/iplayer/subtitles/ng/modav/bUnknown-21b82259-5a4e-40e7-b78b-e8f23463cb7e_m001dhbj_pips-pid-m001dhbj_1681311976912.xml?__gda__=xxx' --output "825. Emeli Sande - Too Many Tickles.en.ttml"
https://github.com/emarsden/dash-mpd-cli is an excellent tool.
I can download from iPkayer, but just the video (with audio) It won't dowload the subtitles:
There's no subtitles for the requested languages
Can reproduce with yt-dlp [email protected] (and Python 3.11.3):
% yt-dlp -vU -F https://www.bbc.co.uk/programmes/m001j638
<snip>
yt-dlp is up to date ([email protected])
[bbc.co.uk] Extracting URL: https://www.bbc.co.uk/programmes/m001j638
[bbc.co.uk] m001j638: Downloading video page
[bbc.co.uk] m001j638: Downloading playlist JSON
[bbc.co.uk] m001j637: Downloading media selection JSON
[bbc.co.uk] m001j637: Downloading media selection JSON
ERROR: [bbc.co.uk] m001j638: bbc.co.uk returned error: selectionunavailable
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 708, in extract
ie_result = self._real_extract(url)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/bbc.py", line 576, in _real_extract
programme_id, title, description, duration, formats, subtitles = self._download_playlist(group_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/bbc.py", line 463, in _download_playlist
version_formats, version_subtitles = self._download_media_selector(programme_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/bbc.py", line 342, in _download_media_selector
self._raise_extractor_error(last_exception)
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/bbc.py", line 327, in _raise_extractor_error
raise ExtractorError(
I am having this issue with https://www.bbc.co.uk/iplayer/episode/p0g0pqbt/radio-1-at-reading-and-leeds-festival-2023-chase-atlantic
get_iplayer can download it fine, fwiw
I'm having this same issue with various videos, including https://www.bbc.co.uk/iplayer/episode/m001hr2v/happy-valley-series-3-episode-5
get_iplayer is downloading it.
