PySceneDetect icon indicating copy to clipboard operation
PySceneDetect copied to clipboard

when splitting scenes, each scene begins with one frame of the previous scene

Open Zero-Gravity58 opened this issue 2 years ago • 6 comments

every scene I am splitting contains the last frame of the previous scene as its first. this defeats my purpose for using the program, of semi-automatically editing down premade content.

would it be possible to add an argument that adds a number of frames to every frame/timecode sent to the splitting program? obviously this would be set to 0 by default. this would make this already convenient tool even more useful.

thank you very much.

Zero-Gravity58 avatar Jan 16 '23 04:01 Zero-Gravity58

I'm really curious as to why this happens. Are you able to share any sample material that exhibits this issue? I want to avoid adding an argument to resolve the issue, since this indicates something is wrong with the underlying video or a possible bug.

What you suggest is certainly possible, but need some more background on what you're trying to achieve and why this is happening. Thanks!

Breakthrough avatar Feb 02 '23 03:02 Breakthrough

In my experience, I have had this issue come up a couple times and it has always been caused by a frame not being decoded correctly due to some error in the video which causes an off-by-one error for all the scene cuts downstream of that improperly decoded frame. I have also seen this same issue a couple times brought up on the discord with the same root cause.

I am not that familiar with how the different backends handle this kind of problem, but is there some way to catch and compensate for this @Breakthrough? I no longer have an example video to test this out on (I long ago deleted and re-downloaded my problem videos), so not sure if there is a way to artificially generate this decoding issue.

wjs018 avatar Apr 15 '23 01:04 wjs018

Your theory definitely has some weight, however both the OpenCV and PyAV backends have switched over to using the timecodes provided by the decoder itself. Timecodes come from the decoded frames, so theoretically a skipped frame shouldn't result in this.

However, in practice it could be different. E.g. the actual thing handling the re-encoding might not handle the frame correctly, or maybe OpenCV/PyAV don't handle this case either. I think without any common reproduction of the issue we're stuck unfortunately.

Breakthrough avatar Jun 30 '23 23:06 Breakthrough

@Zero-Gravity58 can you provide any other information or have you encountered the issue since you originally wrote the bug report? Thanks!

Breakthrough avatar Jul 21 '23 23:07 Breakthrough

@Zero-Gravity58 no times interval between a previous endtime and a following starttime Scene 1: Start 00:00:00.000 / Frame 0, End **00:00:05.000** / Frame 125 Scene 2: Start **00:00:05.000** / Frame 125, End **00:00:10.440** / Frame 261 Scene 3: Start **00:00:10.440** / Frame 261, End 00:00:13.600 / Frame 340 here is a temporary solution /scenedetect/video_splitter.py line 249 for i, (start_time, end_time) in enumerate(scene_list): if i > 0: start_time = start_time + FrameTimecode('00:00:00.200', start_time.get_framerate()) duration = (end_time - start_time)

chj113 avatar Oct 12 '23 05:10 chj113

I believe this to be a bug with ffmpeg. Every other scene has an extra frame, whether split via scenedetect or ffmpeg itself. These were H264 videos in an MKV container. When converted to Y4M, then the split run against the Y4M, the results were as expected and frame-accurate. I'm going to assume it is a problem with how ffmpeg seeks using it's native decoder, though I have not tried decoding with VAAPI or any other hardware decoders yet.

@Breakthrough Would it be possible to take the frames from memory during the scene detection and pipe them to the encoder all in one pass? Even if it consumed massive amounts of memory, many workflows could benefit.

KyleBruene avatar Dec 23 '23 06:12 KyleBruene