tubesync
tubesync copied to clipboard
meeb
Failed downloading videos during the scheduled task
When I add a new source the videos within the cap download just fine. It is only when the scheduled task kicks off that videos run into issues. I'm not sure if it makes a difference, but the /downloads location is on a samba share. I'm running in a docker container.
tubesync_1 | 2022-03-08 18:29:36,029 [tubesync/INFO] [youtube-dl] downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm - 94.7% of 49.12MiB at 1.06MiB/s, 00:02 remaining
tubesync_1 | 2022-03-08 18:29:36,896 [tubesync/INFO] [youtube-dl] downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm - 85.2% of 49.12MiB at 1.15MiB/s, 00:06 remaining
tubesync_1 | 2022-03-08 18:29:38,803 [tubesync/INFO] [youtube-dl] downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm - 89.7% of 49.12MiB at 1.15MiB/s, 00:04 remaining
tubesync_1 | 2022-03-08 18:29:38,893 [tubesync/INFO] [youtube-dl] downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm - 99.6% of 49.12MiB at 894.23KiB/s, 00:00 remaining
tubesync_1 | 2022-03-08 18:29:39,162 [tubesync/INFO] [youtube-dl] finished downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm - 49.12MiB in 00:37
tubesync_1 | 2022-03-08 18:29:40,880 [tubesync/INFO] [youtube-dl] downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm - 94.7% of 49.12MiB at 1.16MiB/s, 00:02 remaining
tubesync_1 | 2022-03-08 18:29:43,854 [tubesync/INFO] [youtube-dl] downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm - 99.7% of 49.12MiB at 848.65KiB/s, 00:00 remaining
tubesync_1 | ERROR: unable to rename file: [Errno 2] No such file or directory: '/downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm.part' -> '/downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm'
tubesync_1 | 2022-03-08 18:29:44,091 [tubesync/INFO] [youtube-dl] finished downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm - 49.12MiB in 00:48
tubesync_1 | 2022-03-08 18:29:49,216 [tubesync/INFO] [youtube-dl] finished downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f251.webm - 10.78MiB in 00:10
tubesync_1 | ERROR: unable to rename file: [Errno 2] No such file or directory: '/downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f251.webm.part' -> '/downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f251.webm'
tubesync_1 | 2022-03-08 18:29:50,808 [tubesync/INFO] [youtube-dl] finished downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f251.webm - 10.78MiB in 00:06
tubesync_1 | ERROR: Conversion failed!
tubesync_1 | 2022-03-08 18:29:51,802 [tubesync/ERROR] Failed to download media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to disk, expected outfile does not exist: /downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv
tubesync_1 | Rescheduling Downloading media for "Inside Kyiv During Putin's invasion"
tubesync_1 | Traceback (most recent call last):
tubesync_1 | File "/usr/local/lib/python3.9/dist-packages/background_task/tasks.py", line 43, in bg_runner
tubesync_1 | func(*args, **kwargs)
tubesync_1 | File "/app/sync/tasks.py", line 404, in download_media
tubesync_1 | raise DownloadFailedException(err)
tubesync_1 | common.errors.DownloadFailedException: Failed to download media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to disk, expected outfile does not exist: /downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv
tubesync_1 | Rescheduling task Downloading media for "Inside Kyiv During Putin's invasion" for 0:00:06 later at 2022-03-08 23:29:57.845785+00:00
tubesync_1 | ERROR: Conversion failed!
tubesync_1 | 2022-03-08 18:29:53,353 [tubesync/ERROR] Failed to download media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to disk, expected outfile does not exist: /downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv
tubesync_1 | Rescheduling Downloading media for "Inside Kyiv During Putin's invasion"
tubesync_1 | Traceback (most recent call last):
tubesync_1 | File "/usr/local/lib/python3.9/dist-packages/background_task/tasks.py", line 43, in bg_runner
tubesync_1 | func(*args, **kwargs)
tubesync_1 | File "/app/sync/tasks.py", line 404, in download_media
tubesync_1 | raise DownloadFailedException(err)
tubesync_1 | common.errors.DownloadFailedException: Failed to download media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to disk, expected outfile does not exist: /downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv
tubesync_1 | Rescheduling task Downloading media for "Inside Kyiv During Putin's invasion" for 0:00:06 later at 2022-03-08 23:29:59.430747+00:00
tubesync_1 | 2022-03-08 18:30:00,343 [tubesync/INFO] Downloading media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to: "/downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv"
tubesync_1 | 2022-03-08 18:30:00,488 [tubesync/INFO] Downloading media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to: "/downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv"
tubesync_1 | 2022-03-08 18:30:02,077 [tubesync/INFO] [youtube-dl] finished downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm - 53.09MiB in ?
tubesync_1 | 2022-03-08 18:30:02,078 [tubesync/INFO] [youtube-dl] finished downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f251.webm - 15.31MiB in ?
tubesync_1 | 2022-03-08 18:30:02,239 [tubesync/INFO] [youtube-dl] finished downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm - 53.09MiB in ?
tubesync_1 | 2022-03-08 18:30:02,240 [tubesync/INFO] [youtube-dl] finished downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f251.webm - 15.31MiB in ?
tubesync_1 | ERROR: Conversion failed!
tubesync_1 | 2022-03-08 18:30:05,043 [tubesync/ERROR] Failed to download media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to disk, expected outfile does not exist: /downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv
tubesync_1 | Rescheduling Downloading media for "Inside Kyiv During Putin's invasion"
tubesync_1 | Traceback (most recent call last):
tubesync_1 | File "/usr/local/lib/python3.9/dist-packages/background_task/tasks.py", line 43, in bg_runner
tubesync_1 | func(*args, **kwargs)
tubesync_1 | File "/app/sync/tasks.py", line 404, in download_media
tubesync_1 | raise DownloadFailedException(err)
tubesync_1 | common.errors.DownloadFailedException: Failed to download media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to disk, expected outfile does not exist: /downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv
tubesync_1 | Rescheduling task Downloading media for "Inside Kyiv During Putin's invasion" for 0:00:21 later at 2022-03-08 23:30:26.084659+00:00
tubesync_1 | ERROR: Conversion failed!
tubesync_1 | 2022-03-08 18:30:05,179 [tubesync/ERROR] Failed to download media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to disk, expected outfile does not exist: /downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv
tubesync_1 | Rescheduling Downloading media for "Inside Kyiv During Putin's invasion"
tubesync_1 | Traceback (most recent call last):
tubesync_1 | File "/usr/local/lib/python3.9/dist-packages/background_task/tasks.py", line 43, in bg_runner
tubesync_1 | func(*args, **kwargs)
tubesync_1 | File "/app/sync/tasks.py", line 404, in download_media
tubesync_1 | raise DownloadFailedException(err)
tubesync_1 | common.errors.DownloadFailedException: Failed to download media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to disk, expected outfile does not exist: /downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv
tubesync_1 | Rescheduling task Downloading media for "Inside Kyiv During Putin's invasion" for 0:00:21 later at 2022-03-08 23:30:26.220883+00:00
tubesync_1 | 2022-03-08 18:30:31,110 [tubesync/INFO] Downloading media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to: "/downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv"
tubesync_1 | 2022-03-08 18:30:31,270 [tubesync/INFO] Downloading media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to: "/downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv"
tubesync_1 | 2022-03-08 18:30:32,457 [tubesync/INFO] [youtube-dl] finished downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm - 53.09MiB in ?
tubesync_1 | 2022-03-08 18:30:32,499 [tubesync/INFO] [youtube-dl] finished downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f251.webm - 15.31MiB in ?
tubesync_1 | 2022-03-08 18:30:32,923 [tubesync/INFO] [youtube-dl] finished downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f247.webm - 53.09MiB in ?
tubesync_1 | 2022-03-08 18:30:32,924 [tubesync/INFO] [youtube-dl] finished downloading: 2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.f251.webm - 15.31MiB in ?
tubesync_1 | ERROR: Conversion failed!
tubesync_1 | 2022-03-08 18:30:35,480 [tubesync/ERROR] Failed to download media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to disk, expected outfile does not exist: /downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv
tubesync_1 | Rescheduling Downloading media for "Inside Kyiv During Putin's invasion"
tubesync_1 | Traceback (most recent call last):
tubesync_1 | File "/usr/local/lib/python3.9/dist-packages/background_task/tasks.py", line 43, in bg_runner
tubesync_1 | func(*args, **kwargs)
tubesync_1 | File "/app/sync/tasks.py", line 404, in download_media
tubesync_1 | raise DownloadFailedException(err)
tubesync_1 | common.errors.DownloadFailedException: Failed to download media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to disk, expected outfile does not exist: /downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv
tubesync_1 | Rescheduling task Downloading media for "Inside Kyiv During Putin's invasion" for 0:01:26 later at 2022-03-08 23:32:01.495730+00:00
tubesync_1 | ERROR: Conversion failed!
tubesync_1 | 2022-03-08 18:30:35,839 [tubesync/ERROR] Failed to download media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to disk, expected outfile does not exist: /downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv
tubesync_1 | Rescheduling Downloading media for "Inside Kyiv During Putin's invasion"
tubesync_1 | Traceback (most recent call last):
tubesync_1 | File "/usr/local/lib/python3.9/dist-packages/background_task/tasks.py", line 43, in bg_runner
tubesync_1 | func(*args, **kwargs)
tubesync_1 | File "/app/sync/tasks.py", line 404, in download_media
tubesync_1 | raise DownloadFailedException(err)
tubesync_1 | common.errors.DownloadFailedException: Failed to download media: dt6mtHEy444 (UUID: 72046e14-227d-488f-bb4a-19b2dc3ffa6c) to disk, expected outfile does not exist: /downloads/video/vicenews/2022-03-07_vicenews_inside-kyiv-during-putins-invasion_dt6mtHEy444_vp9-opus.mkv
tubesync_1 | Rescheduling task Downloading media for "Inside Kyiv During Putin's invasion" for 0:01:26 later at 2022-03-08 23:32:01.887929+00:00
Files
Interesting, that looks like a conversion error (the merging of the video and audio streams with ffmpeg). There are some YouTube formats that require a patch to ffmpeg to merge properly as discussed in other tickets. I'll check your above videos to see if that's one of them. Currently the only way to resolve this would be to manually replace or build a custom version of ffmpeg until some patches are added upstream but ideally I was hoping to avoid that.
Thank you! If that isn't the issue I suspect it could have something to do with me using a network mounted drive for storing the temp files and merging them.
Exactly same problem, first i think that problem in cyrillic names of videos, but as i can see above same problems with non-cyrillic names. But i have strange log entries without filename, only with extension!
2022-03-30 21:11:03,313 [tubesync/INFO] Downloading media: WGD4yL4PR1g (UUID: 6f99e555-1684-4699-bdb4-81d976b7ac47) to: "/downloads/video/later/.mkv" 2022-03-30 21:11:00,770 [tubesync/INFO] Successfully downloaded media: zBffcdgPBk0 (UUID: 10e85986-09b8-401d-bf31-11a99110b842) to: "/downloads/video/later/youtube.mkv" 2022-03-30 21:10:57,889 [tubesync/INFO] [youtube-dl] finished downloading: youtube.f251.webm - 63.09MiB in 00:02 2022-03-30 21:10:55,750 [tubesync/INFO] [youtube-dl] finished downloading: youtube.f303.webm - 542.10MiB in 00:21 2022-03-30 21:11:00,809 [tubesync/INFO] Copying media thumbnail from: /config/media/thumbs/10/10e85986-09b8-401d-bf31-11a99110b842.jpg to: /downloads/video/later/youtube.jpg 2022-03-30 21:11:00,832 [tubesync/INFO] Writing media NFO file to: to: /downloads/video/later/youtube.nfo 2022-03-30 21:11:04,086 [tubesync/ERROR] Failed to download media: WGD4yL4PR1g (UUID: 6f99e555-1684-4699-bdb4-81d976b7ac47) to disk, expected outfile does not exist: /downloads/video/later/.mkv Rescheduling Downloading media for "Татьяна Лазарева: о расколе российского общества, пропаганде и репрессиях @Продолжение следует" common.errors.DownloadFailedException: Failed to download media: WGD4yL4PR1g (UUID: 6f99e555-1684-4699-bdb4-81d976b7ac47) to disk, expected outfile does not exist: /downloads/video/later/.mkv raise DownloadFailedException(err) File "/app/sync/tasks.py", line 404, in download_media func(*args, **kwargs) File "/usr/local/lib/python3.9/dist-packages/background_task/tasks.py", line 43, in bg_runner Traceback (most recent call last):
Ah that's a different issue @hellofaduck - I'll see what I can do for better support for entirely Cyrillic character video titles. TubeSync is currently quite aggressive on stripping out characters (originally designed to strip things like emotes etc.) because quite a few people save their media onto external and older NAS's which are twitchy to say the least with full UTF8 filenames. Not being from a country with a Cyrillic alphabet myself that had minimal testing for some other character sets. I'll probably open a different issue for this later. Thanks for the report and logs.
Thank you! If that isn't the issue I suspect it could have something to do with me using a network mounted drive for storing the temp files and merging them.
I have the same issue, and it only occurs when using a network mounted store. When its directly on the local drive then it works fine.
i have this too when trying to download to a NFS location, interestingly TS can happily create directory structure
EDIT: it appears that the directory structure its creating is octal 755 which doesnt allow it to download to that directory.. if i chmod it to 777 it starts to download.
for reference, using Synolgoy NAS as a storage using NFS Share with RW permission and setting squash to make all users act as admin
@ozhound if tubesync can "mkdir" a new directory but then by default isn't the owner of that directory you must have root squashing or something else enabled on your share? This must be a generic issue with your share and not directly tubesync related, check your NFS permissions and settings, particularly around squashing and UID mapping.
well i have all users mapped to admin. i really dont know how else to make it work as there is no option to map specific users to admin
If all users were properly mapped to admin then all users would be able to write to the directory owned by admin so something isn't working there. It wouldn't be sensible to have tubesync chmod all directories it creates to 777 by default. Try dropping into a shell in tubesync and experimenting with mkdir and touching some files.
It is strange. considering i have *arr containers and transmission downloading and moving files with the same configuration practically continuously. I've moved it from my Proxmox docker container to my docker instance on the synology and used the same user and group id's as i have with my other downloaders that is proven to work and it still cannot create the file
[Error: "Failed to download media: yy8vSLC_8_w (UUID: 30a1b33f-8eea-4737-99e5-4630ec12d6a2) to disk, expected outfile does not exist: /downloads/video/OzhoundRoE/2019-04-06_ozhound-roe_roe-badger-left-me_yy8vSLC_8_w_1080p-avc1-opus-60fps.mkv"](https://tube.ozhound.pw/media/30a1b33f-8eea-4737-99e5-4630ec12d6a2)
That error doesn't necessarily mean a permissions error, it just means that tubesync called yt-dlp to download the file and when complete the expected output file didn't exist on disk, so that could be a permissions issue or it could be something like the download failed. There may be other logs to give more detail on what the error actually was.
If you can drop a shell into the container and create and write to directories fine it's probably not a permissions issue. Does that download magically work if you chmod 777 /downloads/video/OzhoundRoE? Or does it still error?
There's really not much special going on in tubesync that would mean that it creating directories is any different to how other containerised apps you're hosting that work do it, potentially other than UID mapping.
Have you also tried setting the PUID and GUID env vars?
I had this today as well. Used yt-dlp to download that specific video separate
TubeSync version 0.12.1 with yt-dlp version 2023.03.04 (as docker)
and jauderho/yt-dlp DUMB_INIT_VERSION=1.2.5 BUILD_VERSION=2023.03.04 (as docker)
Tubesync log from the container:
023-03-27 14:33:32,976 [tubesync/INFO] Downloading media: _hWPtumg_Js (UUID: e9d53b5d-5569-4c38-a5f9-c0bd6d4f5094) to: "/downloads/video/doublefineprod/2023-02-10_doublefineprod_double-fine-psychodyssey-ep21-much-more-drastic-things_[_hWPtumg_Js].mkv"
ERROR: [youtube] _hWPtumg_Js: Requested format is not available. Use --list-formats for a list of available formats
2023-03-27 14:33:34,061 [tubesync/ERROR] Failed to download media: _hWPtumg_Js (UUID: e9d53b5d-5569-4c38-a5f9-c0bd6d4f5094) to disk, expected outfile does not exist: /downloads/video/doublefineprod/2023-02-10_doublefineprod_double-fine-psychodyssey-ep21-much-more-drastic-things_[_hWPtumg_Js].mkv
Rescheduling Downloading media for "Double Fine PsychOdyssey · EP21: “Much More Drastic Things”"
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/background_task/tasks.py", line 43, in bg_runner
func(*args, **kwargs)
File "/app/sync/tasks.py", line 407, in download_media
raise DownloadFailedException(err)
common.errors.DownloadFailedException: Failed to download media: _hWPtumg_Js (UUID: e9d53b5d-5569-4c38-a5f9-c0bd6d4f5094) to disk, expected outfile does not exist: /downloads/video/doublefineprod/2023-02-10_doublefineprod_double-fine-psychodyssey-ep21-much-more-drastic-things_[_hWPtumg_Js].mkv
Rescheduling task Downloading media for "Double Fine PsychOdyssey · EP21: “Much More Drastic Things”" for 0:40:06 later at 2023-03-27 13:13:40.064236+00:00
That's the same version of yt-dlp stand-alone and embedded in tubesync, both of which are up to date. The error is this line:
ERROR: [youtube] _hWPtumg_Js: Requested format is not available. Use --list-formats for a list of available formats
What this means is that when the media item was indexed the metadata returned by YouTube said a specific format was available, but when it was selected by tubesync to download it wasn't available so the download failed.
Try skipping then unskipping the video, this refreshes the metadata. This isn't specifically an issue with tubesync as it's with broken metadata from YouTube.
yt-dlp worked stand-alone because that just uses a default like --best which has fallback built in if a stream isn't available.
Try skipping then unskipping the video, this refreshes the metadata. This isn't specifically an issue with tubesync as it's with broken metadata from YouTube.
Thanks! will check it
Try skipping then unskipping the video, this refreshes the metadata. This isn't specifically an issue with tubesync as it's with broken metadata from YouTube.
It worked I try to remember this “workaround” (as for example for yt-dlp I have to convert the webp thumbnail to jpg and it is also in command line, so a bit harder to achive the same) Thank you