pinchflat
pinchflat copied to clipboard
kieraneglin
[FR] Move files after download similar to Sonarr
To prevent Plex or other programs from trying to index or access a file during download, allow for downloading to a temporary /Downloads directory. This should prevent excess CPU cycles from multiple programs executing simultaneously.
Once done, then move file to the intended directory for organization. Plex will pick up the file for scanning once moved provided it is set to monitor folder changes.
This should help the program work in tandem with Sonarr for organization.
Would downloading the file with a temporary file extension (youtubevod.tmp) and then move it to the right file extension on complete work too? Plex and other media servers shouldn't pick up that tmp file extension.
That could be a solution. I'm not sure about Plex's folder scanner operation, but it's worth exploring.
Thank you for the report! I've put this on the list of things to look into (:
So this actually would be a pretty easy change, but unfortunately I can't do this until this yt-dlp bug is fixed: yt-dlp/yt-dlp#9445.
I'll leave this open in the meantime and I've also attached a patch of roughly how it would need to be done. Hopefully they are able to fix it soon 🤞
So this actually would be a pretty easy change, but unfortunately I can't do this until this
yt-dlpbug is fixed: yt-dlp/yt-dlp#9445.
Is this why thumbnails are kinda broken? Still having issues with thumbnails not displaying properly in Emby.
@ItsNoted I can't say for sure, but that's my current hunch. I think Emby is being a little too aggressive in its thumbnail caching and it causes some weird behaviours if it picks up the thumbnail when it's in the middle of being converted
Just for fun, I'm going to take a look at that bug myself and see if maybe I can get a PR in to fix it. No promises since yt-dlp is a HUGE and very complicated project, but I'll at least take a look. Wish me luck 🤞
Working through this in https://github.com/yt-dlp/yt-dlp/pull/9774. Hopefully it gets merged soon 🤞
I'm testing if pinchflat can replace my homebrewed yt-dlp orchestration. My scripts use this "two directory" method and it's worked really well.
I'm actually interested in this feature for a different reason. My media storage is on a dedicated NAS but my dockers run on another machine. So doing a bunch of file operations (ffmpeg transcoding in particular) over the network is a lot slower than having a local temp directory and a remote final directory. Splitting the processing and final dirs is how all my other media dockers do things (sabnzbd, deluge, and my custom yt-dlp stuff).
Another benefit I've seen with it is that yt-dlp will sometimes always fail for certain videos part way through. Leaving the artifacts in the temp directory prevents plex from trying to eat something poisonous. With plex's sometimes problematic caching and sometimes fragile media processing, a broken file can be anywhere from annoying to really damaging.
For some reason my NAS doesnt like when I download or transcode from it. My NAS is super old. I also run all my dockers on a separate machine. Ideally I want to save it to a local temp directory on my docker host and once it is downloaded move it to my NAS. I'm subscribing to this issue then :)
@unixrubix If I understood your request correctly, yt-dlp natively supports this functionality. You can use -P temp:<directory> to specify the directory where partially downloaded files will be written. Once the download is complete, the file will be moved to the specified output path.
# Download video as "C:\MyVideos\uploader\title.ext", subtitles as "C:\MyVideos\subs\uploader\title.ext"
# and put all temporary files in "C:\MyVideos\tmp"
$ yt-dlp -P "C:/MyVideos" -P "temp:tmp" -P "subtitle:subs" -o "%(uploader)s/%(title)s.%(ext)s" BaW_jenoz --write-subs
Source: yt-dlp Output Template Examples
@CPSibo and @Ogglord Specifying a local temp path should improve performance when your output path is on a remote network device.
@breakid that's true, but the issue is that there is a bug in yt-dlp where it doesn't report the correct final file path(s) to the app. That is what my PR into yt-dlp resolves and will allow for this functionality
Hopefully, that'll be merged soon. Trying to download to my external drive is a tough task without it
I think I have another usecase for this. I am running the project on a raspberry pi and when using SD-Cards, disk I/O is pretty limited.
I already setup tmpfs for the /tmp diractory in the container. Assuming that yt-dlp/ffmpeg do disk stuff while remuxing etc. this might improve the performance. I have not looked into the file sizes we are talking about here and this is just an experminent.
Also if the tmpfs thing does not work out, offloading this work to a separate "scratch" drive is also an idea i had.
Would love this as well, just had a machine of mine lock up because of high network I/O (I believe). Would be great to be able to output it to the local box and then mux it to the destination on the remote mount.
I have a feeling I can get similar behavior with a custom yt-dlp option. If I get around to it I'll post my results here.
Edit: Tried playing around with the -P option in base-config.txt, could not seem to get it to work. If anyone has any success, please let me know!
+1 here, I'd like to run pinchflat on a more powerful machine and copy the files to my NAS via an SMB connection. I tried it out but I kept getting retries and EOF errors.
@kieraneglin well...that PR just got merged so...any updates on this front
@Yankees4life The yt-dlp PR? It's still open: https://github.com/yt-dlp/yt-dlp/pull/9774