yt-dlp
yt-dlp copied to clipboard
[extractor/twitch] Skip errors in GQL response
IMPORTANT: PRs without the template will be CLOSED
Description of your pull request and other information
This PR fixes the problem where Twitch's GQL API sometimes returns an error before the actual data, breaking the extractor that assumed the video data would be at index 0.
Full json response: https://gist.github.com/elyse0/3b1679b6a0b8d7aa8112f75c1eb52403
Trimmed json
[
{
"errors": [
{
"message": "PersistedQueryNotFound"
}
],
"extensions": {
"durationMilliseconds": 3,
"operationName": "VideoMetadata",
"requestID": "01GSTXCZDDA9GN7ZHBRSB9MP4Y"
}
},
{
"data": {
"video": {
"id": "1697007802",
"moments": {
...
"__typename": "VideoMomentConnection"
},
"__typename": "Video"
}
},
"extensions": {
"durationMilliseconds": 61,
"operationName": "VideoPlayer_ChapterSelectButtonVideo",
"requestID": "01GSTXCZDDA9GN7ZHBRSB9MP4Y"
}
},
{
"data": {
"video": {
"id": "1697007802",
"seekPreviewsURL": "https://d1m7jfoe9zdc1j.cloudfront.net/66e41701aa009d198649_vickypalami_40717337225_1672796209/storyboards/1697007802-info.json",
"__typename": "Video"
}
},
"extensions": {
"durationMilliseconds": 25,
"operationName": "VideoPlayer_VODSeekbarPreviewVideo",
"requestID": "01GSTXCZDDA9GN7ZHBRSB9MP4Y"
}
}
]
Fixes #6308
Template
Before submitting a pull request make sure you have:
- [x] At least skimmed through contributing guidelines including yt-dlp coding conventions
- [x] Searched the bugtracker for similar pull requests
- [x] Checked the code with flake8 and ran relevant tests
In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:
- [x] I am the original author of this code and I am willing to release it under Unlicense
- [ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)
What is the purpose of your pull request?
- [x] Fix or improvement to an extractor (Make sure to add/update tests)
- [ ] New extractor (Piracy websites will not be accepted)
- [ ] Core bug fix/improvement
- [ ] New feature (It is strongly recommended to open an issue first)
With this patch applied, stream titles will randomly fail to be fetched. Which VOD fails to title fetch varies on every run of the for
loop below:
> for /f %a in (vid) do @yt-dlp --get-title %a
Untitled Broadcast
The game in the year of our lord 2023 without FOV options 🤣🤣🤣
BIRTHDAY SPEEDRUNS
Returnal
Untitled Broadcast
SPOOKY SUNDAY - The Karaoke
Untitled Broadcast
Any Goblin Caves Enjoyers?
[ R:\ ]
> for /f %a in (vid2) do @yt-dlp --get-title %a
Untitled Broadcast
Untitled Broadcast
Untitled Broadcast
Elden Ring Any% Glitchless WR Attempts | Atomic Hearts/Rivals Later w/ !team
Untitled Broadcast
Untitled Broadcast
PURE DARKNESS Elden Ring in FIRST PERSON | Horror Mode | !Vods (I won the !KemonoQuest)
Beating Elden Ring in FIRST PERSON | All Remembrances | !Vods (I won the !KemonoQuest)
vid
contents: https://0x0.st/Hs3g.txt
vid2
contents: https://0x0.st/Hs3E.txt
My yt-dlp
environment:
[debug] Command-line config: ['-v']
[debug] User config "C:\Users\garoto\yt-dlp.conf": ['--extractor-args', 'youtube:skip=translated_subs', '--external-downloader-args', '-hide_banner -loglevel warning -stats', '--no-sponsorblock', '--no-mtime', '--no-warnings', '--ignore-errors', '--force-ipv4', '--restrict-filenames', '--merge-output-format', 'mkv', '--sub-lang', 'en.*,pt,pt-BR', '--write-sub', '--embed-subs', '--convert-subs', 'srt', '--format', '(bestvideo[width<=1920][fps<=60][vcodec^=vp9]+bestaudio[acodec=opus])/299+140/137+251/best/bestvideo+bestaudio', '--output', '%(title).70s-%(upload_date)s-%(uploader)s-%(extractor_key)s_%(id).19s.%(ext)s']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version 2023.02.17 [a0a7c01] (pip)
[debug] Python 3.8.2 (CPython AMD64 64bit) - Windows-8.1-6.3.9600-SP0 (OpenSSL 1.1.1d 10 Sep 2019)
[debug] exe versions: ffmpeg 4.0.2, ffprobe 4.0.2
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1782 extractors
Looking at the extractor and error more closely, the problem was that some _OPERATION_HASHES were not up to date
I've updated these hashes and updated some tests
This shoud fix the titles @garoto
This shoud fix the titles @garoto
Tested a bunch of VODs and it seems to be fixed indeed. Thank you!