[Request] Break source into segments for recoverable conversions

[wiznillyp]

@cdgriffith,

Great software! I wrote my own h.265 UHD conversion software before I ever heard of this, but this is much better than mine!

However, I tend to only convert overnight, so I like to have spots in the conversion that are easy to break and recover from. Is there any feature like this in this project?

FYI, To do this in my implementation, I first segment the source file into 3 minute video-only chunks (and a list) with the following parameters in the output string:

-f segment -segment_time_delta 5 -segment_time 180 -segment_list <List_name>

Then, I loop through the resulting files, converting them one at a time and finally concatenating all of the segments back together using the segment list.

I can help implementing this here if it is useful, but I am not sure exactly how to fit it into your UI, especially since this should be able to work in between sessions. This also might help with #227

[cdgriffith]

I am very interested in adding something that could do the batch processing or simple concating as well like https://github.com/cdgriffith/FastFlix/issues/115 ask. Really haven't spent a lot of time on it yet, and really haven't gotten inspiration on how to throw it in without large changes.

If you get ideas, please feel free to add them here!

[MarcoRavich]

@wiznillyp There are also other interesting approaches - such as Av1an one - but needs to be "reimplemented" specifically for FF...

[MarcoRavich]

Well, there are different approaches to video splitting for encoding purposes, so here's some that hopefully may help in its implementation (random order):

• @ValentinMastro's AV1_SPLIT_ENCODE: "First, we run the first pass on the whole source file and use the resulting log file to know which frames are keyframes. Then we cut the source file in splits on each keyframes. They are written in the RAM with the magicyuv codec (lossless, fast encoding/decoding). We run the second pass on each splits in parallel using as many threads as the user provides."

• @martindisch's shepherd: 1. Creates a temporary directory in your home directory. 2. Extracts the audio and encodes it. This is not parallelized, but the time this takes is negligible compared to the video anyway. 3. Splits the video into chunks. This can take relatively long, since you're basically writing the full file to disk again. It would be nice if we could read chunks of the file and directly transfer them to the hosts, but that might be tricky with ffmpeg. 4. Spawns a manager and an encoder thread for every host. The manager creates a temporary directory in the home directory of the remote and makes sure that the encoder always has something to encode. It will transfer a chunk, give it to the encoder to work on and meanwhile transfer another chunk, so the encoder can start directly with that once it's done, without wasting any time. But it will keep at most one chunk in reserve, to prevent the case where a slow machine takes too many chunks and is the only one still encoding while the faster ones are already done. 5. When an encoder is done and there are no more chunks to work on, it will quit and the manager transfers the encoded chunks back before terminating itself. 6. Once all encoded chunks have arrived, they're concatenated and the audio stream added. 7. All remote and the local temporary directory are removed.

• @nashspence's The Video Split Script: A script written in typescript that will use ffmpeg to split a video file into multiple clips based off of either the chapter marker embedded into the video file or an analysis and detection of a visual hard cut.

• @mrbeni's PySceneDetect: a command-line tool, written in Python and using OpenCV, which analyzes a video, looking for scene changes or cuts. The output timecodes can then be used with another tool (e.g. mkvmerge, ffmpeg) to split the video into individual clips. A frame-by-frame analysis can also be generated for a video, to help with determining optimal threshold values or detecting patterns/other analysis methods for a particular video.

• @slhck's Scenecut Extractor: This tool uses the select filter from ffmpeg to determine the scene cut probability of adjacent frames, and allows users to determine which frames (or at which timestamps) the scene cuts happen.

Other:

• @samuelcalegari's FFMpeg-change-scene: Scene Change Detection with Node.js and FFmpeg
• @kieranjol's scenechange: Insert chapter markers for scene changes using Matroska using ffmpeg and mkvpropedit
• @geluso's ffmpeg-scene-extraction: automatically split a large movie file into component scenes
• @bredd's FVDetectScenes: Family Video Scene Detection Tool uses FFMpeg

Hope that inspires !