split-video : why the result of each scene contains the last frame of previous scene

[RaymondWang987]

scenedetect -i /XXX/XXX.mp4 detect-content split-video

I try this mode in some stereo movies why the output clip of each scene always contains the last frame of its previous scene ?

[Breakthrough]

@RaymondWang987 if you also add save-images -m 0 to your command, do the images also contain the wrong frame? This can sometimes happen if input videos have a corrupted frame in them, so if the output of save-images -m 0 is correct, then that is likely the case. Do you also get the same issue if you re-encode the video with ffmpeg before processing?

If you can run the program with debug logs and provide those, that would be very useful:

scenedetect -i /XXX/XXX.mp4 -v debug detect-content split-video

[Breakthrough]

This was supposed to have been fixed in #159, so if you can share any videos exhibiting this issue, that would be very helpful.

Also, if you can try the PyAV backend when processing, see if that helps:

scenedetect -i /XXX/XXX.mp4 -b pyav -v debug detect-content split-video

Lastly, can you share your operating system, and version of ffmpeg you are using? Thanks.

[Breakthrough]

@RaymondWang987 can you provide any more information?

[RaymondWang987]

Thanks for your reply!

I firstly use ffmpeg to extract each chapter of a Blu-ray stereo 3D movie. I find that if the output chapter videos are in mkv format (ffmpeg -i /XXX/fullmovie.mkv -ss XX:XX:XX -to XX:XX:XX chapter_X.mkv), this issue happens in some chapter videos. If those chapter videos are saved in mp4 format (ffmpeg -i /XXX/fullmovie.mkv -ss XX:XX:XX -to XX:XX:XX result.mp4), this issue would not happen.

I am using pyscenedetect and ffmpeg on macOS Monterey 12.4, the version of ffmpeg is：

ffmpeg -version
ffmpeg version N-107466-g009ef35d38-tessus Copyright (c) 2000-2022 the FFmpeg developers built with Apple clang version 11.0.0 (clang-1100.0.33.17) configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvmaf --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-version3 --pkg-config-flags=--static --disable-ffplay libavutil 57. 30.100 / 57. 30.100 libavcodec 59. 39.100 / 59. 39.100 libavformat 59. 29.100 / 59. 29.100 libavdevice 59. 8.101 / 59. 8.101 libavfilter 8. 46.100 / 8. 46.100 libswscale 6. 8.101 / 6. 8.101 libswresample 4. 8.100 / 4. 8.100 libpostproc 56. 7.100 / 56. 7.100

[RaymondWang987]

I will conduct more experiments by adding save-images -m 0 or -b pyav with mkv and mp4 videos as your suggestions. I will report the results later.

[Breakthrough]

If those chapter videos are saved in mp4 format (ffmpeg -i /XXX/fullmovie.mkv -ss XX:XX:XX -to XX:XX:XX result.mp4), this issue would not happen.

This points to some frame failing to be decoded, which can throw off the internal timekeeping. This can cause all frame numbers to get shifted, which is likely why you're seeing extra frames in split videos. This doesn't necessarily mean the video is corrupt, but it does mean that OpenCV isn't handling the video correctly, which is not something that can be solved within PySceneDetect.

If you run scenedetect with --verbosity debug, you should see some errors in the logs pointing to this issue ("Frame failed to decode"). If this is the case, let me know and I will make sure it shows an error by default (currently it only shows an error if more than one frame fails decoding). Using -b pyav may help as it uses a different decoder / timekeeping method, so if you can test that and let me know if that helps the issue, that would be great.

Can you provide the output of the following commands:

1. scenedetect -i /XXX/XXX.mkv -v debug detect-content
2. scenedetect -i /XXX/XXX.mkv -b pyav -v debug detect-content

Thank you.

[Breakthrough]

Closing issue for now, feel free to reply with more information if/when possible.