VodRecovery
VodRecovery copied to clipboard
ItIckeYd
Option to download only valid segments of VOD
Recently I came across for the first time a VOD that is still available but certain segments are missing.
I tried downloading the m3u8 with yt-dlp but that ended up failing and I couldn't play it back in my media player either. Would you consider adding support to build your own m3u8 file from only the valid segments?
For example if the user opts to check for valid segments and there are segments that aren't valid, prompt the user to create and save a new m3u8 with only the valid links.
@Infinitay I have this like 90% working in testing but still requires some work to ensure it works as intended in all cases. It will take some time but will be implemented in the near future.
Related question: With using an extension like "video download helper" we can get links to vods that are online & we have normal access to on twitch site and with those links downloads never fail even if the vods have muted segments but if we get the link through recovery programs downloads fail as soon as we hit the first muted segment. Muted segments are still online so why is it like this? And why can't recovery tools can't get the same links as the extension? Will we always have to download the hundreds/thousands of segments individually and merge them ourselves if the vod has muted segments?
@blackesteagle I'm not sure how those programs work as I don't use them personally but by reading your question it seems like those tools use the twitch api. My script does not use twitch api or any service that twitch provides.. It is a standalone script. It uses the endpoints of twitch servers to retrieve vods/clips by generating a formatted m3u8 link based on the basic vod information provided by any of the analytical sites that are noted in the README.
This is my assumption.. I could be wrong.
Since extensions like "video download helper" work on almost all sites that have a video playing on it((except on youtube on chrome) I assumed they use page source, api would be too specific but I don't know. Copying this link through the extension for instance: `
https://usher.ttvnw.net/vod/1548134551.m3u8?allow_source=true&p=2675136&play_session_id=859eb8c655656045cdbaec5d58f6bda7&player_backend=mediaplayer&playlist_include_framerate=true&reassignments_supported=true&sig=7e20fdb16d680108ebf21a750fe8037a3c7bbda6&supported_codecs=avc1&token=%7B%22authorization%22%3A%7B%22forbidden%22%3Afalse%2C%22reason%22%3A%22%22%7D%2C%22chansub%22%3A%7B%22restricted_bitrates%22%3A%5B%5D%7D%2C%22device_id%22%3A%22wKqKggRq3MuAKnP6aOOkR3wVPoXXo9kW%22%2C%22expires%22%3A1659356372%2C%22https_required%22%3Atrue%2C%22privileged%22%3Afalse%2C%22user_id%22%3A93723833%2C%22version%22%3A2%2C%22vod_id%22%3A1548134551%7D&cdm=wv&player_version=1.12.0
makes JDownloader list all the available quality versions of the vod that are failproof. I just don't understand why muted segments that are still online causing this problem with the links we get via recovery programs.
Related question: With using an extension like "video download helper" we can get links to vods that are online & we have normal access to on twitch site and with those links downloads never fail even if the vods have muted segments
twitch mutes segments for dmca reasons - so it would not make any sense to have actual non muted version of streams/segments still online at this point there was a short time after the stream ended when you could still grab it without muted segments - but their muting is done faster and faster and now the vod has muted segments identified like 15-30 min after a 12 hour stream ends
for a short while 1-4 hours - definitely less than 24 hours - even the muted segments are often available to be grabbed manually from a browser window (with automatic like jdownloader they often pop out as offline - maybe too many grabbed too fast)
the link posted above for cohhhhcarnage is exactly the same we get from script - although only the highest quality version this one in particular does not have any muted segments and the results are exactly the same from this addon/link and script
the problem is with muted vods - example https://www.twitch.tv/videos/1538236868?t=3h11m51s - in browser it will continue https://vod-secure.twitch.tv/7b86a62743165a78af31_jinnytty_41015701915_1658395282/chunked/index-dvr.m3u8 when you paste a m3u8 link of a muted vod without distinct segments to vlc you get the player to stop playing/recording same in Jdownloader it will stop at that moment - so these link are not at all failproof
Sorry for the confusion I wasn't asking about downloading/playing the muted segments as normal(with sound) and posting that non-muted cohh vod didn't help I guess but through the extension you can also get muted vods that are failproof like that vod you posted for example:
`
https://usher.ttvnw.net/vod/1538236868.m3u8?allow_source=true&p=6899659&play_session_id=6586323be696aff3453e2d339c43a719&player_backend=mediaplayer&playlist_include_framerate=true&reassignments_supported=true&sig=e7c885f7aa226b6c2792eb0efe656cc26ced830e&supported_codecs=avc1&token=%7B%22authorization%22%3A%7B%22forbidden%22%3Afalse%2C%22reason%22%3A%22%22%7D%2C%22chansub%22%3A%7B%22restricted_bitrates%22%3A%5B%5D%7D%2C%22device_id%22%3A%22wKqKggRq3MuAKnP6aOOkR3wVPoXXo9kW%22%2C%22expires%22%3A1659425781%2C%22https_required%22%3Atrue%2C%22privileged%22%3Afalse%2C%22user_id%22%3A93723833%2C%22version%22%3A2%2C%22vod_id%22%3A1538236868%7D&cdm=wv&player_version=1.12.0
With these links muted segments still play without sound in VLC(I use mpc but should act the same)/downloads don't fail.
so the link you gave for robbaz vod on 20220720 that starts muted is https://dgeft87wbj63p.cloudfront.net/04e714a7dbe3dbcf7ebb_robbaz_39644442920_1658332398/chunked/index-muted-SWKT3LZUHO.m3u8 and below is the script one https://dgeft87wbj63p.cloudfront.net/04e714a7dbe3dbcf7ebb_robbaz_39644442920_1658332398/chunked/index-dvr.m3u8 The first one indeed starts playing in vlc same as in browser You are onto something here So this playlist is the superior one and the more convenient one we should try and grab from twitch - index-muted-SWKT3LZUHO At this point beyond my skills but worth pointing out for a a smarter person to maybe investigate
Yeah, need to get that master link/manifest somehow and through it the index-muted links. Anyway I just wanted to point that out to see if its easy to implement or not, apparently not :)
Just note that if you run ffmpeg -i LINK -c copy OUTPUT.mp4 it will just download the valid chunks anyway.
I've had some cases where that did not work. It would be interesting to see how edited VODs would work in instances where a channel editor removes part of a VOD, and if it would remain as valid or not.
I've had some cases where that did not work. It would be interesting to see how edited VODs would work in instances where a channel editor removes part of a VOD, and if it would remain as valid or not.
Honestly that doesn't make sense to me. ffmpeg will try to reach each of the segments in the playlist and ignore the inaccessible ones, you just need to provide it with the unmuted playlist. But anyway I also implemented in my personal shell script a function to recreate the playlist file with only the valid segments, so it is possible to watch the VOD normally through VLC/MPV without the need to download it first.
From recovering so many vods i think this functionality is not that useful as most vods are usually either all segments are there or only 20% there. Also you don't actually know what part of the vod will be available could be a part where the streamer is afk. I think it's highly unlikely you will find the part of the stream you are looking for.
This functionality is now available.. Users are now able to download M3U8 links.
The script downloads only the available segments. If the script comes across an invalid segment it continues and the vod will skip to the next available segment once download is complete.
Closing issue now.
Reopened Issue as new functionality was added to generate a new M3U8 file with only valid segments included making any vod playable as long as there is atleast 1 segment available.