tubesync
tubesync copied to clipboard
meeb
[bug] can't download files which filename will exceed 255 characters
From my added sources, there are 14 tasks stuck, because they fail to download:
ERROR: unable to open for writing: [Errno 36] File name too long: '/downloads/video/Kochou Cocoa/2021-07-18 - 【3Dio#ASMR 】同接１５０超えるまでやめれません！耳はむ多めで甘々でデレデレになっちゃう最高の癒し♡たくさん甘えて？♡耳フー♡ please Request【#胡蝶ここあ Vtuber】 (DdMVc7-Ks8A).f299.mp4.part'
The original video has a character length of 98 characters but because they are unicode, this translates to 225 ASCII characters. After tubesync adds the pre- and suffix as well as the part file extensions it easily exceeds 255 characters.
There needs to be a check of the the maximum resulting character limit (when there are more parts to fetch the resulting filename gradually grows) and the Video title needs to be trimmed.
As a workaround I managed to download the videos separately and added them as external videos.
What file system are you using for your
/downloadsmount?
A SMB docker mount, from a TrueNAS Share, so in short ZFS which has a hard limit of 255 chars.
Maybe the best solution would be to first download the video with only its video ID as filename and rename it after it's completed to a max 255 chars filename.
I'll look at implementing a truncation for video titles. This is a relatively rare event because YouTube limit titles to 100 characters so to be over 255 bytes once encoded from UTF8 into bytes would be pretty unusual. Just your video title alone expands from 55 characters to 165 bytes.
Truncating the variable used to generate the title component of the filename formatter to be within 255 chars (including the rest of the filename format template) seems the most logical option.
I'll look at implementing a truncation for video titles. This is a relatively rare event because YouTube limit titles to 100 characters so to be over 255 bytes once encoded from UTF8 into bytes would be pretty unusual. Just your video title alone expands from 55 characters to 165 bytes.
I ran into this issue 7 times 😅 (but I also downloaded 2000+ videos). All videos were from Japanese channels, hence the Unicode issue.
Truncating the variable used to generate the title component of the filename formatter to be within 255 chars (including the rest of the filename format template) seems the most logical option.
Sounds good, but this could still cause an issue with very long videos exceeding 1000 parts, because the filename can grow during downloading
Sounds good, but this could still cause an issue with very long videos exceeding 1000 parts, because the filename can grow during downloading
Yeah I was going to pad on sufficient space for this, an extra 12 chars spare should do it to cover the part number and extension with dots.
Yeah I was going to pad on sufficient space for this, an extra 12 chars spare should do it to cover the part number and extension with dots.
While possible, I'd prefer a more versatile approach. Some filesystem allow for longer filenames and setting a fixed length for padding could remove more characters than necessary. Maybe expose the value of the maximum filename character to an environment variable so that it can be overridden? And renaming the file after it was downloaded would prevent the removal of more characters than necessary.
I experienced the same issue when including {title_full} in the filename for some videos.
Under log of the container, the below error message is repeated 3xx times by increasing the number behind "Frag"
ERROR: unable to open for writing: [Errno 36] File name too long: '/downloads/cache/
Meanwhile, under the folder on nas, there's a PART file and a YTLD file with the video's long name which seems the filename can be such long... a little confused here as the error message says "File name too long" but the temp file has the same amount of characters.
@AC13139 Yeah that doesn't make any sense as clearly your path isn't too long. What file system is your /downloads volume on? It's probably not something in-container our our end to be honest.
I'm almost certain yt-dlp has a format option that could help with this.
Anyway, the idea of downloading to a short filename then renaming the file would not be too terrible now that the automated renaming has been implemented.
We don't need to worry about the yt_dlp configuration, if we pass it a filename that's just the date, key, and extension. It can add as many parts to that as it needs to during the download.
Then we only need to ensure our final format based filename is not over the limit.
