tubesync
tubesync copied to clipboard
meeb
Specific Items failed to download
I've got a series of videos failing to download. They return an "expected outfile doesn't exist" error in the error item of the tasks page.
Running on Unraid, TubeSync version is 0.10.0
Logs: https://pastebin.com/1JxzUfQN
Was coming here to post the same exact error. I have the same OS and same tubesync version. Been experiecing this issue for over a week.
My workaround is to delete the 1 mkv and 2 webm files from the specific download folder, then click on the failed task in tubesync and click skip media. It then downloads and converts just fine. The other weird thing is this doesn't happen to the same sources. One day 1 source is fine, the next day it's not. I've tried deleting everything and starting from scratch with no luck.
Huh, that's odd. I had tried deleting the mkv/webm files and requeing all tasks (which leads me to an error page every time without fail), but I wasn't able to get them to download.
I also have problems with all of the sources being queued up, so I've only noticed it with these few specific videos from the few sources they come from and only these ones.
This is almost certainly the same as #164. The work-around that @superjedi27 mentions I suspect is fixing the downloads purely because it's retrying the download after enough time. This seems to be an issue with YouTube reporting a format as available, but they aren't ready yet so the video file parts 404 for several hours.
On paper this should be an upstream yt-dlp issue, however I'm mulling over a sensible way to handle it in TubeSync. The options are:
- Do nothing, maybe increase download retry attempts so the downloads eventually work
- Fall back to a different format if video parts 404
The problem with 1 is it might take an extra day for videos to download for no clear reason. The problem with 2 is it may not be obvious why one specific video is in an undesired resolution of different codec for no clear reason.
Additionally detecting this error seems to be quite annoying as the YouTube-is-returning-a-404-for-DASH-video-parts error isn't (as far as my limited checking so far as found) returned over the API so I guess you'd have to parse logs to detect it. All TubeSync really does is kick off a download via the yt-dlp API then once the call completes check if the expected output file exists. A failed download is the file doesn't exist, a successful download is the file does exist. It was implemented in this way so that pretty much any download issues weren't TubeSync's issue as it was just sitting on youtube-dl or yt-dlp to do all the actual work.
This probably occurred now after the switch to yt-dlp because it impersonates a mobile client and I would suspect that the mobile clients are expected to all handle this issue themselves if I were to make a guess.
If it commonly becomes required that TubeSync has to more intelligently work out specific failure events and have different logic for different failures it could get quite messy.
I'll check the upstream reports and hopefully this will be fixed upstream without anything in TubeSync needing to be patched over.
@HStep20 - not reset all tasks, click on the specific failed task after deleting the media and hit skip media, it re-queues the single task so don’t worry.
@meeb - if it helps you, one of my failed tasks were from a video uploaded 9 hours prior and another 1 hour prior. Both were failed and both downloaded just fine after deleting the media and resetting the single task. The video from an hour prior was only 6 minutes long so maybe it finished doing whatever on YouTube’s end and became available? I read the other thread, I’ll try letting the errors sit and see if they ever resolve, if so its just a slight inconvenience.
@superjedi27 yeah that matches my own testing. The retries have exponential back-off so if the download fails for a few hours it might take up to a day to eventually download. Let me know if it hasn't resolved itself by tomorrow.
As an update, I tried out the suggestion to delete the media, skip and requeue all of them, but it continued having issues. So I stopped the app, did the suggested database cleanup I've seen in other issue threads due to jobs not queuing (since I have that problem too), and started it back up to try again - same result.
5 of the 6 that continue erroring out are from 2015, and one is from 2016, so I don't think it's a 'New on Youtube' problem. I can also watch the video perfectly fine on youtube, so I don't think its on their end.
After some more digging this does look like an upstream issue. TubeSync relies on the format that the library says is available being available. I may have to look at some extended fallback methods to work around this. Primarily, this affects DASH containers it seems.
I don’t know if this is possible and it may be a negative for some by wasting data. If it is possible maybe make it an option? But after X amount of attempts or X amount of hours delete the 3 files and have it retry? Or would that be a lot of work and messiness you were talking about before by TubeSync needing to be more intelligent and parse?
Yes that would require TubeSync to have logic to attempt to detect what the failure type is before attempting work-arounds. At the moment a "failure" could be anything, internet problems, YouTube is down, your connection is dead, ran out of hard drive space, anything. The 3 partial files are deleted if the download eventually succeeds anyway so they're not a massive concern to keep tidy, they're tracked by yt-dlp.
Detecting this one specific failure out of other possible failures would require the messy bit.