tubesync icon indicating copy to clipboard operation
tubesync copied to clipboard

Some files don't download and throw an error: "failed to download media to disk, expected outfile does not exist"

Open JustTooKrul opened this issue 1 year ago • 6 comments

Some media sources seem to work, but the majority do not. In the "Tasks" section of the UI, it throws an error:

Downloading media for "

", attempted 2 times Error: "Failed to download media: <ID> (UUID: <UUID>) to disk, expected outfile does not exist: /downloads/video/<channel>/<date, channel, title, and ID>1080p-vp09-opus-60fps.mkv" Task will be retried at 2024-03-25 15:11:22 <p>In the logs, I get this:</p> <p>2024-03-25 18:39:51,740 [tubesync/ERROR] Failed to download media: <ID> (UUID: <UUDI>) to disk, expected outfile does not exist: /downloads/video/<channel>/<date, channel, title, and ID>_premium-vp09-opus.mkv Rescheduling Downloading media for "<title>" Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/background_task/tasks.py", line 43, in bg_runner func(*args, **kwargs) File "/app/sync/tasks.py", line 476, in download_media raise DownloadFailedException(err) common.errors.DownloadFailedException: Failed to download media: <id> (UUID: <UUDI>) to disk, expected outfile does not exist: /downloads/video/<channel>/<date, channel, title, and ID>_premium-vp09-opus.mkv Rescheduling task Downloading media for "<title>" for 0:01:26 later at 2024-03-25 22:41:17.744073+00:00

JustTooKrul avatar Apr 16 '24 14:04 JustTooKrul

Can you share the YouTube video ID? That possibly looks like an issue with the selected stream being premium.

meeb avatar Apr 17 '24 06:04 meeb

I added six channels and this happens with five of them... I will share the full entries for the last few, but a popular channel, Wendover Productions, is a good example of one that doesn't work.

Downloading media for "Why China's Economy is Finally Slowing Down", attempted 2 times Error: "Failed to download media: ascPhiXcpss (UUID: 28cbd32e-6793-4222-a51e-967c77bdc3e7) to disk, expected outfile does not exist: /downloads/video/wendoverproductions/2024-03-21_wendover-productions_why-chinas-economy-is-finally-slowing-down_ascPhiXcpss_premium-vp09-opus.mkv"

Downloading media for "The Increasing Reality of War in Space", attempted 2 times Error: "Failed to download media: V0DmliiUFHk (UUID: f604f2c8-216c-4eb8-9ae9-606fa76e4a5e) to disk, expected outfile does not exist: /downloads/video/wendoverproductions/2024-03-06_wendover-productions_the-increasing-reality-of-war-in-space_V0DmliiUFHk_premium-vp09-opus.mkv"

I suspect the default quality assumptions may be causing an issue? That being said, I also set the the options to "Get next best resolution or codec instead" ... Here are the quality options I set:

Source resolution : 2160p (4K) Source video codec : VP9 Source audio codec : OPUS Prefer 60FPS? : yes Prefer HDR? : yes   Output extension : mkv Fallback : Get next best resolution or codec instead

And it happened on a few videos where the "premium" identified didn't occur:

Downloading media for "Containerization: The Most Influential Invention That You've Never Heard Of", attempted 2 times Error: "Failed to download media: F-ZskaqBshs (UUID: f2477c82-7f10-4d58-b7e6-a09209f45988) to disk, expected outfile does not exist: /downloads/video/wendoverproductions/2015-12-08_wendover-productions_containerization-the-most-influential-invention-that-youve-never-heard-of_F-ZskaqBshs_1080p-vp09-opus.mkv"

Downloading media for "How To Install a Pipeline Under a Railroad", attempted 3 times Error: "Failed to download media: RK1J8kC1sEY (UUID: 281b301a-ed29-46ca-bc09-2a87ca61dbc1) to disk, expected outfile does not exist: /downloads/video/practicalengineeringchannel/2024-02-20_practical-engineering_how-to-install-a-pipeline-under-a-railroad_RK1J8kC1sEY_2160p-vp09-opus.mkv"

(As you can see, I started with a bunch of educational content to watch while offline...)

JustTooKrul avatar Apr 19 '24 17:04 JustTooKrul

Any luck with this? I have tried changing the quality to 1080p and the audio format to mp4 (I think it says mp4a?) and some additional videos downloaded, but there are still some errors. As an example:

Downloading media for "Will There Ever Be Another World's Tallest Building?", attempted 3 times Error: "Failed to download media: T4CCY9RJeD4 (UUID: 72d97e63-d3f6-4295-bfdb-8596b7421216) to disk, expected outfile does not exist: /downloads/video/theb1m/2019-12-11_the-b1m_will-there-ever-be-another-worlds-tallest-building_T4CCY9RJeD4_1080p-vp09-mp4a.mkv" Task will be retried at 2024-05-01 14:07:59

Downloading media for "Top 20 Projects Completing in 2020", attempted 3 times Error: "Failed to download media: LZQsMSaeLO8 (UUID: 9b86d9d4-6cf4-42ac-a711-4bf14d76d59b) to disk, expected outfile does not exist: /downloads/video/theb1m/2020-01-01_the-b1m_top-20-projects-completing-in-2020_LZQsMSaeLO8_1080p-vp09-mp4a.mkv" Task will be retried at 2024-05-01 14:07:59

JustTooKrul avatar May 01 '24 18:05 JustTooKrul

There isn't really enough information here for me to be able to to tell you what the issue is. The error you're pasting here, the "expected outfile does not exist" one, is just the final check error. This just means "tubesync called yt-dlp, and when yt-dlp completed the expected downloaded file didn't exist on the disk". This can be caused by pretty much any failure, for example anything from YouTube has blocked your IP to your internet connection had an outage to you ran out of disk space. It's impossible to know what the actual error is without additional logs. Try and find some other errors in your log of what actually went wrong with the download.

Also, try "skipping" then "unskipping" (buttons at the bottom of the media page) a failed item of media. That will cause tubesync to refresh the media items metadata and try again.

meeb avatar May 02 '24 03:05 meeb

Tried the "skipping and unskipping" as well as rebuilding the entire task list, and neither works. Changing the formats and the quality downloads more videos, but still has a significant number of failures.

Isn't there an option for automatically getting the best available quality if the pre-selected options aren't available? Also, FWIW, TubeArchivist gets everything from these channels, but that's obviously not helpful given the lack of any naming support and the proprietary way it deals with files... I only mention it, because it seems to be doable to get these videos with yt-dlp and some combination of options / parameters.

JustTooKrul avatar May 16 '24 19:05 JustTooKrul

I’m running into this same issue. Since the final error is misleading, are you able to preserve the actual error that would be useful in debugging and show that in the UI instead?

https://www.youtube.com/watch?v=IdZhXQQ6xAo is an example of the video mine is failing on with

Matched formats | Combined: no match Audio: 251 (exact match) Video: 248 (fallback)

Mine downloaded 338 successfully and failed on 48 repetitively even after many hard rests of the tasks.

ptarjan avatar Jun 27 '24 04:06 ptarjan

I currently have 4 videos stuck with this error. I've tried restarting my docker container, and checked I'm running the latest version, but they are still retrying and failing, and I've been able to download the videos manually with yt-dip without any error.

Let me know if there's any logs I can supply, or a release with some extra debugging I can run, which will help track down the problem.

RichardHyde avatar Jul 10 '24 21:07 RichardHyde

After digging through the TubeSync logs this morning the videos failing with this error all seem to return the warning nsig extraction failed: You may experience throttling for some formats.

There is a fixed yt-dlp issue which mentions this warning and is included in the latest release.

Can we get a TubeSync release with this versions of yt-dlp to see if it fixes this issue?

RichardHyde avatar Jul 11 '24 10:07 RichardHyde

@RichardHyde there was a tubesync release yesterday that includes yt-dlp version 2024.07.09, update to :latest to get it.

meeb avatar Jul 11 '24 10:07 meeb

Fantastic! I should have seen that, doh!

I'll update my container and see if it helps

RichardHyde avatar Jul 11 '24 10:07 RichardHyde

So far, so good. Two of the five videos I had retrying to download due to the "outfile does not exist" error have now downloaded after updating to the latest TubeSync release, the other three are set to be attempted again this evening (UK time). I'll let them retry as scheduled rather than forcing them, just in case that skews the result.

If you've been getting this error try the latest version and report back.

RichardHyde avatar Jul 11 '24 12:07 RichardHyde

The latest release seems to have solved this issue for me. All my errored videos have downloaded successfully, and there haven't been any other instances of the error since.

RichardHyde avatar Jul 11 '24 18:07 RichardHyde

I'll close this issue for now, feel free to comment if you are still experiencing issues.

meeb avatar Aug 03 '24 10:08 meeb

I think I'm affected by something related to this, but it seems they fixed it on 204.08.06

https://github.com/yt-dlp/yt-dlp/pull/10668

danmarce avatar Aug 07 '24 03:08 danmarce

I'll pull in the updated yt-dlp shortly.

meeb avatar Aug 07 '24 05:08 meeb

Yep this worked.

Some downloads were broken so I set those to skip, deleted the partially downloaded media (only the video was downloaded), unskip, and it worked beautifully. Thank you!

danmarce avatar Aug 07 '24 16:08 danmarce