ytarchive icon indicating copy to clipboard operation
ytarchive copied to clipboard

Video fragments are not downloaded

Open qwer-lives opened this issue 2 months ago • 15 comments

For some streams, only audio is downloaded and the video fragments are stuck at 0 (please ignore the random special characters, they're unrelated to the issue):

ytarchive 0.5.0-556e712
2025/12/10 15:06:15 INFO: [32mLoaded cookie file /tmp/r38w4n9m/2025-12-10 - 23h06m15s - hebich (Hebi) - youtube - 헤비와 함께하는 마인크래프트 1일차 [2025.12.10] - 1080p60_c.txt[0m[K
2025/12/10 15:06:16 Channel: Hebi.
2025/12/10 15:06:16 Video Title: 헤비와 함께하는 마인크래프트 1일차 [2025.12.10]
2025/12/10 15:06:16 Selected quality: 1080p60 (VP9)
2025/12/10 15:06:16 Stream started at time 2025-12-10T14:05:59+00:00
2025/12/10 15:06:16 INFO: [32mStarting download to /home/red/archiving/output/youtube/Hebi/Nflr7HEzoQI__1587276480/2025-12-10 - 23h06m15s - hebich (Hebi) - youtube - 헤비와 함께하는 마인크래프트 1일차 [2025.12.10] - 1080p60.f140.ts[0m[K
2025/12/10 15:06:16 INFO: [32mStarting download to /home/red/archiving/output/youtube/Hebi/Nflr7HEzoQI__1587276480/2025-12-10 - 23h06m15s - hebich (Hebi) - youtube - 헤비와 함께하는 마인크래프트 1일차 [2025.12.10] - 1080p60.f303.ts[0m[K
Video Fragments: 0; Audio Fragments: 1; Max Fragments: 20; Max Sequence: 20; Total Downloaded: 17.93KiB[K
Video Fragments: 0; Audio Fragments: 2; Max Fragments: 20; Max Sequence: 20; Total Downloaded: 35.85KiB[K
Video Fragments: 0; Audio Fragments: 3; Max Fragments: 20; Max Sequence: 20; Total Downloaded: 53.76KiB[K
Video Fragments: 0; Audio Fragments: 4; Max Fragments: 20; Max Sequence: 20; Total Downloaded: 72.04KiB[K
Video Fragments: 0; Audio Fragments: 5; Max Fragments: 21; Max Sequence: 21; Total Downloaded: 89.95KiB[K
...
Video Fragments: 0; Audio Fragments: 15057; Max Fragments: 15056; Max Sequence: 15056; Total Downloaded: 264.68MiB[K
Video Fragments: 0; Audio Fragments: 15057; Max Fragments: 15056; Max Sequence: 15056; Total Downloaded: 264.68MiB[K
Video Fragments: 0; Audio Fragments: 15057; Max Fragments: 15056; Max Sequence: 15056; Total Downloaded: 264.68MiB[K
2025/12/10 19:19:45 INFO: [32mDeleting file /home/red/archiving/output/youtube/Hebi/Nflr7HEzoQI.f140.state[0m[K

2025/12/10 19:19:45 Download Finished

Error opening input files: Invalid data found when processing input
2025/12/10 19:19:45 ERROR: [31mExecute returned code 183. Something must have gone wrong with ffmpeg.[0m[K
2025/12/10 19:19:45 ERROR: [31mThe .ts files will not be deleted in case the final file is broken.[0m[K
2025/12/10 19:19:45 ERROR: [31mFinally, the ffmpeg command was either written to a file or output above.[0m[K

It seems to happen only with VP9 streams. Interestingly, I have tried both removing the --vp9 flag and adding the `--h264 flag but ytarchive seems to keep picking the VP9 stream:

~/Downloads > ytarchive "https://www.youtube.com/watch?v=XbbpVgjzRUM&pp=0gcJCSkKAYcqIYzv" best --no-wait --no-frag-files --merge --threads 4 --retry-frags 6 --mkv --verbose
ytarchive 0.5.0-556e712
2025/12/10 21:57:24 Channel: DrDisRespect
2025/12/10 21:57:24 Video Title: 🔴LIVE - DR DISRESPECT - TARKOV 1.0 - DEAL OR NO DEAL
2025/12/10 21:57:24 Selected quality: 1440p60 (VP9)
2025/12/10 21:57:24 Stream started at time 2025-12-10T16:30:52+00:00
Video Fragments: 0; Audio Fragments: 37; Total Downloaded: 719.57KiB^C
2025/12/10 21:57:31 WARNING: User Interrupt, Stopping download...
~/Downloads > ytarchive "https://www.youtube.com/watch?v=XbbpVgjzRUM&pp=0gcJCSkKAYcqIYzv" best --no-wait --no-frag-files --merge --threads 4 --retry-frags 6 --mkv --verbose --h264
ytarchive 0.5.0-556e712
2025/12/10 21:58:11 Channel: DrDisRespect
2025/12/10 21:58:11 Video Title: 🔴LIVE - DR DISRESPECT - TARKOV 1.0 - DEAL OR NO DEAL
2025/12/10 21:58:11 Selected quality: 1440p60 (VP9)
2025/12/10 21:58:11 Stream started at time 2025-12-10T16:30:52+00:00
Video Fragments: 0; Audio Fragments: 43; Total Downloaded: 479.95KiB^C
2025/12/10 21:58:13 WARNING: User Interrupt, Stopping download...

Setting --h264 at the start seems to work but not with best (since I suspect that one only has VP9), instead of falling back automatically to the best h264 stream it prompts again:

red Dec 10 22:04 ~ > ytarchive --h264 "https://www.youtube.com/watch?v=XbbpVgjzRUM&pp=0gcJCSkKAYcqIYzv" best --no-wait --no-frag-files --merge --threads 4 --retry-frags 6 --mkv --verbose
ytarchive 0.5.0-556e712
2025/12/10 22:04:30 Channel: DrDisRespect
2025/12/10 22:04:30 Video Title: 🔴LIVE - DR DISRESPECT - TARKOV 1.0 - DEAL OR NO DEAL
2025/12/10 22:04:30 The qualities you selected ended up unavailable for this stream
2025/12/10 22:04:30 You will now have the option to select from the available qualities
Available video qualities: audio_only, 144p, 240p, 360p, 480p, 720p60, 1080p60, 1440p60, best
Enter desired video quality: 1080p60
2025/12/10 22:04:35 Selected quality: 1080p60 (h264)
2025/12/10 22:04:35 Stream started at time 2025-12-10T16:30:52+00:00
Video Fragments: 7; Audio Fragments: 21; Total Downloaded: 10.18MiB^C
2025/12/10 22:04:38 WARNING: User Interrupt, Stopping download...

I've received reports from different people experiencing the same too, so it looks like youtube changed something.

So basically a few issues:

  • VP9 streams (at least some) don't seem to work
  • --h264 only works if it's set before the quality selection
  • When using best and the best quality is not H264, --h264 doesn't automatically fall back to the best available H264 quality

Update: On dev branch the recording works but 1440p format is not there, so I'm assuming YouTube just removed VP9 non-adaptive formats recently or something similar.

qwer-lives avatar Dec 10 '25 21:12 qwer-lives

First thing's first, all options have to come before the URL and quality selector. That's how Go's opt parser does things. Setting an option after normal inputs will not parse them. Second, seems I and others are hit by this now as well. When I first saw this issue opened, I had checked and my stream archives were still getting 1440p VP9. Looks like Youtube pulled VP9 from the DASH manifest, so now we're left with only h264 at a max of 1080p60. In fact it seems like they don't offer h264 for anything above 1080p anymore in general.

The sad reality is, I don't think I'll be able to implement anything for grabbing the data via SABR or the adaptive formats. They require more circumvention than I am likely willing to figure out, even with yt-dlp as a reference, in part because it likely requires actually executing some of Youtube's JS. God I fucking hate Youtube honestly.

Kethsar avatar Dec 11 '25 23:12 Kethsar

I'm fine with forcing h264 and getting a max of 1080p60, but would setting best quality be advised in this scenario?

When using best and the best quality is not H264, --h264 doesn't automatically fall back to the best available H264 quality

I worry that this might happen if I keep it to best quality with the --h264 flag.

RisingFog avatar Dec 11 '25 23:12 RisingFog

Though that is a genuine bug I will eventually fix, frankly there's no reason for using --h264 if you don't explicitly want it anyway. It will automatically choose h264 if VP9 is not available when using --vp9. My immediate guess is Youtube was in some kind of transition when you had these issues where they may have still listed the VP9 qualities in the DASH manifest, but they were not actually available.

My default options are ytarchive --vp9 --no-frag-files --threads 5 --add-metadata -w -t. The most recent stream I archived, which is the first to encounter Youtube removing VP9 from dash, grabbed the 1080p60 h264 stream instead.

Kethsar avatar Dec 12 '25 00:12 Kethsar

After I post that I notice the stream I am currently archiving has VP9 So now I am not sure what is going on really. Still only 1080p60 but that's just the quality she streams at.

Kethsar avatar Dec 12 '25 00:12 Kethsar

It seems intermittent from the various streams I've been archiving, but lately I've been noticing a lot more of the streams are not serving VP9 (I was wondering why I was seeing errors where only the audio recorded).

TBH I'm not a stickler for codecs/formats as long as I can get a good quality recording of a stream, so if sticking to h264 is more stable with YouTube, I will continue to use it.

RisingFog avatar Dec 12 '25 00:12 RisingFog

Yeah I just don't know anymore. A new stream from the usual person that does 1440p60 is now serving that at VP9 this time despite her stream from just 3 hours ago not having it. Did I mention I fucking hate Youtube?

Kethsar avatar Dec 12 '25 00:12 Kethsar

The Game Awards stream is following the same issue. Audio fragments downloaded only. I've always liked ytarchive since it actually can dl 2160p VP9 video from yt lives compared to when I last used yt-dlp (h264 only for lives).

Kethsar, I too relate to this sentiment. I fucking hate consolidated monopolies that make themselves the only option and proceed to extract as much capitol as possible from it's inelastic user base by removing features or making them paid only. Hell it seems more then anything a desire for locking down their content for god knows what "security reasons".

UwUChox avatar Dec 12 '25 00:12 UwUChox

I tested again and now all streams I tested have working VP9 at 1440p60 so maybe YouTube was/is testing their removal from the DASH manifest these past few days. However I would expect them to remove them in the near future if they're testing it now.

Also, thanks for the context on the arguments, just setting --vp9 works to avoid this issue. I wonder if a release would be warranted if YouTube goes through with tihs change, since the current release does not filter out the adaptive formats.

qwer-lives avatar Dec 12 '25 07:12 qwer-lives

Hello, Kethsar!

Indeed, now in the dash manifest that ytarchive receives there are no formats '1440p' and '2160p'. I think that to get these formats you need to edit the Go's parser if you can do it. These formats are available on the server. There is such a tool: https://getlate.dev/tools/youtube-live-downloader. It doesn't use a dash manifest, but allows you to download a 5 second snippet. If there are fragments, then there may be a manifest.

AAZENON avatar Dec 14 '25 14:12 AAZENON

I've experienced this a few times as well, including with a livestream yesterday that was broadcast in 2160p that I normally download in 1440p. ytarchive was unable to get any video frames at all, even with specifying the -h264 or -vp9 options, or 1080p resolution. I was able to get 1080p with yt-dlp, but yt-dlp couldn't identify any formats above 1080p.

danarnold avatar Dec 18 '25 23:12 danarnold

Someone in a chat with me finally mentioned the video fragments not downloading as it was happening, so I was able to figure that part out. I forgot to push to master after removing the code that attempts to use formats in adaptiveFormats since we can no longer access those. I have done that now, and the newest pre-release should allow you to use -vp9 without trying to grab data that it cannot/does not exist.

Kethsar avatar Dec 19 '25 15:12 Kethsar

Sounds good, I'll give it a try. To be clear, is --vp9 required or will it automatically pull the appropriate format without specifically adding that flag?

danarnold avatar Dec 19 '25 16:12 danarnold

--vp9 is only required if you want to continue having a chance at grabbing VP9 should they sometimes pop up in the DASH manifest. I'm not sure that will ever happen since it seems the rollout that was being done to remove it is complete, but you never know. It doesn't hurt, at least.

Kethsar avatar Dec 19 '25 16:12 Kethsar

I can confirm that the prerelease version works with streams that previously didn't grab video. I'm able to get 1080p h.264 video.

danarnold avatar Dec 23 '25 05:12 danarnold

--vp9 is only required if you want to continue having a chance at grabbing VP9 should they sometimes pop up in the DASH manifest. I'm not sure that will ever happen since it seems the rollout that was being done to remove it is complete, but you never know. It doesn't hurt, at least.

unfortunately even with '--vp9' command 2160p was unable to be grabbed, only been able to get 1080p

yy0901 avatar Dec 25 '25 10:12 yy0901