Skip frame for Adaptive

[MattFanto]

Description of Problem & Solution I'm trying to process many high fps videos and unfortunately the video processing would be too low for our use case, I saw the skip frames option. Unfortunately it can't be applied to the Adaptive algorithm due to incompatibility with the mandatory scene stats. I was wondering if it would be possible to create a more generic implementation of skip frames that can be used with the Adaptive detector.

Proposed Implementation: Eventually adding a parameter to VideoManager, would allow skipping the frames directly there, creating an abstraction for the downstream detector so that the least amount of changes would be required.

Alternative Solutions: The alternative solution in my case would be to reduce the FPS eagerly, with some libraries like FFmpeg. Unfortunately, that would be much more resource-consuming compared to the lazy approach above.

[Breakthrough]

I'm pursuing turning AdaptiveDetector into an online algorithm for the next release. This would also mean it no longer requires a StatsManager. It does mean some API changes as the way callbacks work right now (they assume a cut can be detected immediately), but that's something that should be resolved for the long term regardless.

I'll use this as a tracking issue for that feature, but hope to have this done for v0.6.1.

In general have you had to tweak the parameters you're using with Adaptive? If this proceeds as planned, I want to verify that the defaults are reasonable for most use cases, as I would like to essentially make what AdaptiveDetector does now the new default for v0.6.1.

Thanks!

Edit: Updated new approach (previously was to merge with ContentDetector, but they should remain separate, and usage examples point people to that as a default in the next release).

[MattFanto]

Hi @Breakthrough thanks a million for your reply, great news looking forward to trying the new version.

Regarding your question: at least to me the default are reasonable

[Breakthrough]

I managed to make the transition, and AdaptiveDetector now doesn't require a StatsManager. Here's a beta version you can try if you're interested in testing (can be installed using pip):

scenedetect-0.6.1.dev0.tar.gz

Couple things remain as TODOs, but most of them are for the CLI and code organization - the above build should work just fine as is.

• [ ] Had to change short option of -w/--frame-window to -f, using it instead for --weights
  • consider a different name for --weights or if this breaking change should proceed
• [x] Decide if this functionality should be merged with ContentDetector or remain separate
  • Arguments for merging:
    • it can be turned off by setting window size to zero
    • reduces code duplication when handling CLI and config file options
    • doesn't affect main score calculation
    • most users already are using ContentDetector/ detect-content so will experience improvement when updating
  • Arguments against merging
    • may cause changes in behavior for end users if turned on by default
    • detectors need to be refactored anyways
    • changing the documentation is easier / lower risk, and this can always be pursued in the future
  • Final decision: No merge
• [x] ~If merge is made, mark AdaptiveDetector/detect-adaptive as deprecated, otherwise~ update examples to use AdaptiveDetector/detect-adaptive instead of ContentDetector/detect-content
• [x] Change default min_content_val value to 18 (current default is 15) due to results of local testing

[MattFanto]

Hi @Breakthrough , sorry for the late reply I just came back from holiday and thanks a million for the wheel. I'm adapting some parameter since I was using v0.5 and some changes are required to migrate to v0.6.x, in any case I already checked skip frame and it's working great. I'm available also to test a release candidate as soon as it is available

[MattFanto]

Hi @Breakthrough , all working good I have adapted the params etc.. and I didn't see any regression, the performance improvement in our case is excellent, thanks a lot! May I ask you when are you going to release it to PyPI? I'm wondering from a deployment perspective if I should commit the wheel or I can just wait for the package in PyPI and install via pip

[Breakthrough]

I'm not sure on an exact timeline, but I hope to have a pre-release of v0.6.1 within the next couple weeks on PyPI. The final v0.6.1 release will probably be near the end of ~September~ October. Unfortunately I can't commit to any specific dates yet as I'm busy with IRL stuff the next couple ~weeks~ months.

[MattFanto]

Thanks for the info, no problem at all. In any case I can use the wheel