Recap Detection

[jumoog]

find a way thats not CPU/GPU heavy

[jumoog]

https://www.primevideotech.com/computer-vision/automatically-detecting-recaps-introductions-and-credits-in-content-at-scale

[jumoog]

Here are some approaches for recap detection in TV series:

• Scene Classification with Machine Learning: This method involves training a machine learning model to classify video scenes based on features like fast cuts, flashbacks, and voiceovers, which are common in recaps. Tools like TensorFlow or PyTorch can be used for this purpose.

• Audio Fingerprinting: Techniques like audio fingerprinting can be used to identify recurring music or sound effects that indicate recaps. This might involve generating fingerprints for the intros and outros of known recap segments and then comparing them to the episodes.

• Leveraging Subtitles/Closed Captions: If available, subtitles or closed captions can offer clues about recaps. Look for phrases like "Previously on" or "Last time" that often precede recaps.

• Pre-existing Recap Information: Some TV series might have information about recaps readily available online (e.g., fan forums, episode summaries). You can utilize this to identify recap segments.

[jumoog]

Okay the english subtitle has <i>Previously on</i> and ends with a long blackframe

[jumoog]

https://github.com/Hellowlol/bw_plex does this

previously on, last season, last episode

Jellyfin extracts subtitles on demand, not during indexing. So we have to extract the subtitles ourselves using ffprobe and ffmpeg.

[AbandonedCart]

Don't forget checking chapters. As far as anime goes, they do label the recap (if they label chapters anyway).

[jumoog]

I guess we can use https://github.com/jellyfin/jellyfin-plugin-subtitleextract to extract the subtitles during scan