tubesync icon indicating copy to clipboard operation
tubesync copied to clipboard

Media Stuck Downloading

Open waxill opened this issue 3 months ago • 16 comments

I have been seeing issues with my tubesync install for a while now which I put down to the large sqlite DB. So I finally made the switch to a postgreSQL set up and rebuilt my entire set up and began to add the sources back in 1 by 1.

But, nothing will download. It indexes everything, but when a download starts they just get stuck "downloading" and I see these errors in the logs;

tubesync | 2025-09-26 09:57:27,752 [tubesync/INFO] waiting for errors: 429 (00:00:00): KHH2344TDwfeY tubesync | 2025-09-26 09:57:36,341 [tubesync/WARNING] [youtube] KH34H24TDwfeY: Signature extraction failed: Some formats may be missing tubesync | player = https://www.youtube.com/s/player/a61444a1/player_ias.vflset/en_US/base.js tubesync | Please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U tubesync | 2025-09-26 09:57:37,264 [tubesync/WARNING] [youtube] KHH3424TDwfeY: Some web client https formats have been skipped as they are missing a url. YouTube is forcing SABR streaming for this client. See https://github.com/yt-dlp/yt-dlp/issues/12482 for more details [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR: [download] Got error: HTTP Error 403: ForbiddenRROR] ERROR:

tubesync | 2025-09-26 09:58:58,659 [tubesync/INFO] waiting for errors: 429 (00:00:00): KH34H24TDwfeY

tubesync | WARNING: [youtube] KHH24TDwfeY: Signature extraction failed: Some formats may be missing tubesync | player = https://www.youtube.com/s/player/a61444a1/player_ias.vflset/en_US/base.js tubesync | Please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U tubesync | WARNING: [youtube] KHH24TDwfeY: Some web client https formats have been skipped as they are missing a url. YouTube is forcing SABR streaming for this client. See https://github.com/yt-dlp/yt-dlp/issues/12482 for more details tubesync | 192.168.0.10 - - [26/Sep/2025:10:01:01 +0100] "GET /tasks HTTP/1.1" 200 3023 "http://192.168.99.20:4848/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36" tubesync | ERROR: unable to download video data: HTTP Error 403: Forbidden tubesync | [2025-09-26 10:04:58,999] ERROR:huey:189:Unhandled exception in task fb684597-9b8f-4745-b4a1-43bf814563ca. tubesync | Traceback (most recent call last): tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 3516, in process_info tubesync | partial_success, real_download = self.dl(fname, new_info) tubesync | ^^^^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 3254, in dl tubesync | return fd.download(name, new_info, subtitle) tubesync | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/downloader/common.py", line 480, in download tubesync | ret = self.real_download(filename, info_dict) tubesync | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/downloader/http.py", line 371, in real_download tubesync | establish_connection() tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/downloader/http.py", line 122, in establish_connection tubesync | ctx.data = self.ydl.urlopen(request) tubesync | ^^^^^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 4239, in urlopen tubesync | return self._request_director.send(req) tubesync | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/networking/common.py", line 117, in send tubesync | response = handler.send(request) tubesync | ^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/networking/_helper.py", line 194, in wrapper tubesync | return func(self, *args, **kwargs) tubesync | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/networking/common.py", line 359, in send tubesync | return self._send(request) tubesync | ^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/networking/_requests.py", line 357, in _send tubesync | raise HTTPError(res, redirect_loop=max_redirects_exceeded) tubesync | yt_dlp.networking.exceptions.HTTPError: HTTP Error 403: Forbidden tubesync | tubesync | During handling of the above exception, another exception occurred: tubesync | tubesync | Traceback (most recent call last): tubesync | File "/app/sync/youtube.py", line 458, in download_media tubesync | return y.download([url]) tubesync | ^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 3660, in download tubesync | self.__download_wrapper(self.extract_info)( tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 3633, in wrapper tubesync | res = func(*args, **kwargs) tubesync | ^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 1657, in extract_info tubesync | return self.__extract_info(url, self.get_info_extractor(key), download, extra_info, process) tubesync | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 1668, in wrapper tubesync | return func(self, *args, **kwargs) tubesync | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 1824, in __extract_info tubesync | return self.process_ie_result(ie_result, download, extra_info) tubesync | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 1883, in process_ie_result tubesync | ie_result = self.process_video_result(ie_result, download=download) tubesync | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 3065, in process_video_result tubesync | self.process_info(new_info) tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 187, in wrapper tubesync | return func(self, *args, **kwargs) tubesync | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 3543, in process_info tubesync | self.report_error(f'unable to download video data: {err}') tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 1126, in report_error tubesync | self.trouble(f'{self._format_err("ERROR:", self.Styles.ERROR)} {message}', *args, **kwargs) tubesync | File "/usr/local/lib/python3.11/dist-packages/yt_dlp/YoutubeDL.py", line 1065, in trouble tubesync | raise DownloadError(message, exc_info) tubesync | yt_dlp.utils.DownloadError: ERROR: unable to download video data: HTTP Error 403: Forbidden tubesync | tubesync | The above exception was the direct cause of the following exception: tubesync | tubesync | Traceback (most recent call last): tubesync | File "/usr/local/lib/python3.11/dist-packages/huey/api.py", line 394, in _execute tubesync | task_value = task.execute() tubesync | ^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/huey/api.py", line 834, in execute tubesync | return func(*args, **kwargs) tubesync | ^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/usr/local/lib/python3.11/dist-packages/django_huey/init.py", line 23, in inner tubesync | return fn(*args, **kwargs) tubesync | ^^^^^^^^^^^^^^^^^^^ tubesync | File "/app/sync/tasks.py", line 1014, in download_media_file tubesync | format_str, container = media.download_media() tubesync | ^^^^^^^^^^^^^^^^^^^^^^ tubesync | File "/app/sync/models/media.py", line 1026, in download_media tubesync | download_youtube_media(self.url, format_str, self.source.extension, tubesync | File "/app/sync/youtube.py", line 466, in download_media tubesync | raise YouTubeError(f'Failed to download for "{url}": {e}') from e tubesync | sync.youtube.YouTubeError: Failed to download for "https://www.youtube.com/watch?v=xxx": ERROR: unable to download video data: HTTP Error 403: Forbidden

waxill avatar Sep 26 '25 09:09 waxill

You picked an unfortunate time. YouTube very recently made changes to make downloading more difficult.

Pull the latest image.

tcely avatar Sep 26 '25 11:09 tcely

I have the latest image, and I'm also having this issue. 403 forbidden on most if not all downloads.

davidancalagon avatar Sep 26 '25 18:09 davidancalagon

Same issue for me. Guessing it's YouTube making it hard to download and another sync/update of yt-dlp is needed.

necromancyr avatar Sep 26 '25 23:09 necromancyr

[!TIP] You can test that theory using the upgrade script.

Command
TUBESYNC_DEBUG=True /app/upgrade_yt-dlp.sh
Example output
root@9d47030f5673:/app# TUBESYNC_DEBUG=True /app/upgrade_yt-dlp.sh
Please report any issues that you have encountered before updating yt-dlp.

This is a tool to assist developers with debugging YouTube issues.
It should not be used as an alternative to updating container images!
Requirement already satisfied: yt-dlp in /usr/local/lib/python3.11/dist-packages (2025.9.23)
Collecting yt-dlp
  Using cached yt_dlp-2025.9.26-py3-none-any.whl (3.2 MB)
Installing collected packages: yt-dlp
  Attempting uninstall: yt-dlp
    Found existing installation: yt-dlp 2025.9.23
    Uninstalling yt-dlp-2025.9.23:
      Successfully uninstalled yt-dlp-2025.9.23
Successfully installed yt-dlp-2025.9.26

tcely avatar Sep 27 '25 07:09 tcely

Seems to be working now (at least, SOMETHING downloaded while nothing would previously).

I also setup POT, which wouldn't work last night - but now appears to be also given no error re: 502 being thrown in the below. 127.0.0.1 - - [27/Sep/2025:10:09:49 -0400] "GET /ping HTTP/1.1" 200 77 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36" 127.0.0.1 - - [27/Sep/2025:10:09:49 -0400] "POST /get_pot HTTP/1.1" 200 291 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36"

Also, just because I'm paranoid - does setting TUBESYNC_DEBUG that way (via console) persist over container restarts? I didn't think it would but...just want to be sure.

necromancyr avatar Sep 27 '25 14:09 necromancyr

Also, just because I'm paranoid - does setting TUBESYNC_DEBUG that way (via console) persist over container restarts? I didn't think it would but...just want to be sure.

Setting an environment variable before a command in bash doesn't even persist to the next command.

You will need to set that variable from docker for it to persist across restarts or for newly created containers.

tcely avatar Sep 27 '25 14:09 tcely

Thanks - both for the way to update yt-dlp and for the info (always learning!

And seems to be working well right now - wasn't just a one time thing. Multiple videos downloading over time.

necromancyr avatar Sep 27 '25 15:09 necromancyr

No dice for me here. Did the update to yt-dlp-2025.9.26 and still getting 403 forbidden on video downloads.

davidancalagon avatar Sep 27 '25 18:09 davidancalagon

No luck for me either :/ Just updated to the new version and I also get the same error: yt_dlp.utils.DownloadError: ERROR: unable to download video data: HTTP Error 403: Forbidden.

Maybe not related (IDK) but the media it's stuck at has a weird video format proposition (TV-S?):

ID: 278 , 144p, TV-S (256x144), fps:25, video:VP9 @63.213k
ID: 394 , 144p, TV-S (256x144), fps:25, video:AV1 @37.626k
ID: 133 , 240p, TV-S (426x240), fps:25, video:AVC1 @84.278k
ID: 242 , 240p, TV-S (426x240), fps:25, video:VP9 @76.863k
ID: 395 , 240p, TV-S (426x240), fps:25, video:AV1 @69.746k
ID: 134 , 360p, TV-S (640x360), fps:25, video:AVC1 @167.069k
ID: 18 , 360p, TV-S (640x360), fps:25, video:AVC1 @474.535k , audio:MP4A 44,100Hz [fr]
ID: 243 , 360p, TV-S (640x360), fps:25, video:VP9 @158.184k
ID: 396 , 360p, TV-S (640x360), fps:25, video:AV1 @133.351k
ID: 135 , 480p, TV-S (854x480), fps:25, video:AVC1 @275.748k
ID: 244 , 480p, TV-S (854x480), fps:25, video:VP9 @237.342k
ID: 397 , 480p, TV-S (854x480), fps:25, video:AV1 @222.48k
ID: 136 , 720p, TV-S (1280x720), fps:25, video:AVC1 @461.13k
ID: 247 , 720p, TV-S (1280x720), fps:25, video:VP9 @434.181k
ID: 398 , 720p, TV-S (1280x720), fps:25, video:AV1 @393.066k
ID: 137 , 1080p, TV-S (1920x1080), fps:25, video:AVC1 @1601.623k
ID: 248 , 1080p, TV-S (1920x1080), fps:25, video:VP9 @801.341k
ID: 399 , 1080p, TV-S (1920x1080), fps:25, video:AV1 @639.727k (matched) 

KalyaSc avatar Sep 27 '25 19:09 KalyaSc

So, I also added Proof-Of-Origin Token setup with bgutil-ytdlp-pot-provider - I'm guessing it's a combo of the two, because it was NOT working last night when I was trying to get things going.

Basically, run that in another container, and just set the env variables TUBESYNC_POT_PORT and TUBESYNC_POT_IPADDR to it.

necromancyr avatar Sep 27 '25 22:09 necromancyr

I also experienced this issue and updating yt-dlp (by 3 days) fixed it for me for a while. I've triggered a rebuild of the last commit which should update yt-dlp in the container. This is probably going to need more upstream fixes to resolve properly as it's not a tubesync issue directly. yt-dlp will be updated in the container as and when an updated release is available.

meeb avatar Sep 28 '25 07:09 meeb

Maybe not related (IDK) but the media it's stuck at has a weird video format proposition (TV-S?):

It's unlikely to be related, but I can't rule it out.

TV-S should be referring to the client (tv_simply) being used by yt-dlp for anyone who was wondering.

tcely avatar Sep 28 '25 07:09 tcely

Maybe I was a bit impatient ? :x Today it worked for the file mentioned above, however the available formats are just plain different:

 ID: 91 , WEB-S (256x144), fps:25.0, video:AVC1 @171.852k , audio:MP4A 0Hz [fr]
ID: 92 , WEB-S (426x240), fps:25.0, video:AVC1 @312.535k , audio:MP4A 0Hz [fr]
ID: 93 , WEB-S (640x360), fps:25.0, video:AVC1 @804.444k , audio:MP4A 0Hz [fr]
ID: 18 , 360p, TV (640x360), fps:25, video:AVC1 @474.535k , audio:MP4A 44,100Hz [fr]
ID: 94 , WEB-S (854x480), fps:25.0, video:AVC1 @1355.687k , audio:MP4A 0Hz [fr]
ID: 95 , WEB-S (1280x720), fps:25.0, video:AVC1 @2568.543k , audio:MP4A 0Hz [fr]
ID: 96 , WEB-S (1920x1080), fps:25.0, video:AVC1 @4698.627k , audio:MP4A 0Hz [fr] 

I guess I should just chill and let it do its thing.

Thanks again for your help and project !

KalyaSc avatar Sep 30 '25 10:09 KalyaSc

You can access formats through each client. So, the more clients that can be made to work, the better.

A lot of clients are requiring Proof-of-Origin Tokens now.

For web clients we have a plugin to generate these that can be setup. Also, when a cookie file is used the token will be cached by a plugin between tasks.

YouTube will continue to change things and yt-dlp is making changes to compensate. The available formats will change over time.

tcely avatar Sep 30 '25 14:09 tcely

I've triggered a rebuild of the last commit which should update yt-dlp in the container.

Any chance of putting a new tag on the image? Using latest is problematic with declarative deployment methods.

yt-dlp will be updated in the container as and when an updated release is available.

When should we expect yt-dlp to be updated? If it's updated automatically, what is the new image for?

georgek avatar Oct 01 '25 09:10 georgek

Any chance of putting a new tag on the image? Using latest is problematic with declarative deployment methods.

We do periodic releases, there will shortly be a release for October for example. However, you really don't want to use explicit tags as you'll be updating the tag potentially multiple times a week. This is annoying for declarative deployments, however it's not really sensible to create new tags so regularly.

When should we expect yt-dlp to be updated? If it's updated automatically, what is the new image for?

yt-dlp is not updated in the container automatically. A new image is built and released with the updated yt-dlp included in it.

meeb avatar Oct 01 '25 09:10 meeb