Failed download due to conversion error, suspect it is not

[ankenyr]

I made a playlist with a single video which is giving me some issues. https://www.youtube.com/watch?v=-RdOwhmqP5s&list=PLMLJ9CnbaPl2uDfXrPZipTfM4XHUN5DNL&index=1

I am using the following template {yyyymmdd}_{title}_[{key}].{ext} Downloading 8k, VP9, OPUS, Prefer 60FPS and HDR, output as MKV and fallback is "Get next best resolution or codec instead" and copy thumbnails and write NFO.

It ends up failing stating it was a conversion error

[download] 100% of 25.48MiB in 00:17
[download] 100% of 25.48MiB in 00:17                 2021-11-01 19:16:09,484 [tubesync/INFO] [youtube-dl] finished downloading: 20211012_newtons-fractal-which-newton-knew-nothing-about_[-RdOwhmqP5s].f251.webm - 25.48MiB in 00:17

ERROR: Conversion failed!
2021-11-01 19:16:11,346 [tubesync/ERROR] Failed to download media: -RdOwhmqP5s (UUID: a35a342b-b8d9-4009-b15d-b4c15632504e) to disk, expected outfile does not exist: /downloads/video/foo/20211012_newtons-fractal-which-newton-knew-nothing-about_[-RdOwhmqP5s].mkv
Rescheduling Downloading media for "Newton's Fractal (which Newton knew nothing about)"
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/background_task/tasks.py", line 43, in bg_runner
    func(*args, **kwargs)
  File "/app/sync/tasks.py", line 404, in download_media
    raise DownloadFailedException(err)
common.errors.DownloadFailedException: Failed to download media: -RdOwhmqP5s (UUID: a35a342b-b8d9-4009-b15d-b4c15632504e) to disk, expected outfile does not exist: /downloads/video/foo/20211012_newtons-fractal-which-newton-knew-nothing-about_[-RdOwhmqP5s].mkv
Rescheduling task Downloading media for "Newton's Fractal (which Newton knew nothing about)" for 0:00:06 later at 2021-11-02 02:16:17.365789+00:00
ERROR: Conversion failed!

I feel that this may be an issue inside of tubesync itself as I can, from inside of the docker container, issue the same yt-dl command and it works.

[meeb]

What's the 50 or so log lines above that snippet? Specifically around the downloading of the audio stream.

Does the download work in the container with yt-dlp with the exact same formats specified with -f? Most of these issues are because TubeSync directly specifies the format which YouTube reports is available, but then isn't. yt-dlp works on the command like with the default +bestvideo or similar because it usually requests a different format. You can get the format codes off the TubeSync interface to test.

These sort of issues are generally unlikely to be TubeSync because by itself it really doesn't do much, it just specifies the requirements and then waits for yt-dlp and ffmpeg to complete, then checks the output file exists. This error is just that the output file doesn't exist so either the download process failed or that the conversion or format merge failed.

The two most common causes so far have been the format that is reported as available by yt-dlp isn't actually downloadable for some reason from YouTube, or that the format downloaded has a really weird container format which needs an additional ffmpeg plugin to be added to the container.

[ankenyr]

someone@somewhere:/downloads/video/bar# yt-dlp --no-colors --age-limit=99 --ignore-errors --cache-dir='/config/cache' --add-metadata --format '315+251' --merge-output-format mkv -o '2021-10-12_foo_newtons-fractal-which-newton-knew-nothing-about_-RdOwhmqP5s_2160p-vp9-opus-60fps.mkv' -k -q https://www.youtube.com/watch?v=-RdOwhmqP5s
root@41f6484a820d:/downloads/video/bar# ls
2021-10-12_foo_newtons-fractal-which-newton-knew-nothing-about_-RdOwhmqP5s_2160p-vp9-opus-60fps.f251.webm
2021-10-12_foo_newtons-fractal-which-newton-knew-nothing-about_-RdOwhmqP5s_2160p-vp9-opus-60fps.f315.webm
2021-10-12_foo_newtons-fractal-which-newton-knew-nothing-about_-RdOwhmqP5s_2160p-vp9-opus-60fps.mkv

I think I got the command correct from your outputs. You can see the log output here

This could still be a yt-dlp issue but I am unsure. Wondering if you can see something wrong with my command above that points this back at yt-dlp?

[meeb]

Excellent thanks for the details. I'll have a test to see why the same command is failing over the API. From your logs it looks like it might well be just ffmpeg failing to merge the streams for some reason.

[ankenyr]

So I was searching through the issues on youtube-dlp when I found this issue https://github.com/yt-dlp/yt-dlp/issues/871 Seems it is an ffmpeg issue. I rebuilt the docker container with the version they custom built and the error went away. Would you be willing to change the docker file to include this version of ffmpeg?

[meeb]

Thanks @ankenyr this is fantastic. This would have occurred when I switched to distro-packaged ffmpeg in preparation for multi-arch builds and the particular formats required to trigger it meant I never encountered this myself in testing. I'm happy to bundle whatever is a sensible version of ffmpeg providing it's multi-arch and kept up to date automatically. I'll check out the yt-dlp auto-builds, I must admit I'd not noticed them until now.

[meeb]

Note to my future self: https://github.com/yt-dlp/FFmpeg-Builds/wiki/Latest

[ankenyr]

No worries, I did not know about this until last night. Not sure what multi-arch means in this context.

[meeb]

Multi-arch is just TubeSync that works on Raspberry Pi (ARMv7) or other different architectures. I switched to distro-packaged ffmpeg as they handle this for you, previously I was manually installed ffmpeg that just works for x86 and this was blocking TubeSync from running on other devices.

[pukkandan]

The yt-dlp builds are the same as BtbN builds. I am not sure whether they are multi-arch as you want. Either way, the important part of the build is this patch. Any build that applies this patch should work around the postprocessing issue

[meeb]

Thanks @pukkandan, most useful.

[meeb]

This should be resolved in :latest.