Rotate scene output images according to video metadata

[markbaumgarten]

Description of Problem & Solution Videos recorded using cell phones plays fine in software like VLC(because vlc reads the video metadata).

Using opencv2 to output thumbs from files that are recorded 'sideways' outputs thumbs that are rotated - because opencv ignores the metadata. By using ffprobe to detect rotations this problem can be mitigated.

I am using pyscenedetect in an effort to do video deduplication/recognition - and it is essential that I am able to detect/fix rotated videos.

Media Examples: Sorry - will try to provide examples later

Proposed Implementation: Please have a look here:

https://github.com/markbaumgarten/PySceneDetect/commit/6a3a8a36eb67bddbdaef1b80eac3482739676213

...still a noob with regards to creating a proper pull...or merge...? request. Apologies for any inconvenience. Hints for me to do this the right way are very welcome.

Alternative Solutions: None I know of

[Breakthrough]

Hi @markbaumgarten;

This is definitely something I would like to add to PySceneDetect, so thank you for providing a reference implementation. The only thing I would add to the code you wrote is an additional check to detect if ffprobe is available.

I am also wondering, do you know of any other tools (or preferably, native Python libraries) which can also determine this information?

Thank you!

[markbaumgarten]

I have added a check for ffprobe presence now. Not sure how to inform the user he should install ffprobe.

I personally wrap all of my software in docker containers so stuff like this is not much of an issue in my end.

There are ffprobe pip packages, but when trying to use them I have had some issues I did not manage to fix.

I believe that my code is only valid for python 3.3+ - hoping this is not an issue.

Best regards and many thanks for creating scenedetect.

Mark

On Tue, Nov 19, 2019 at 2:17 AM Brandon Castellano [email protected] wrote:

Hi @markbaumgarten https://github.com/markbaumgarten;

This is definitely something I would like to add to PySceneDetect, so thank you for providing a reference implementation. The only thing I would add to the code you wrote is an additional check to detect if ffprobe is available.

I am also wondering, do you know of any other tools (or preferably, native Python libraries) which can also determine this information?

Thank you!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Breakthrough/PySceneDetect/issues/134?email_source=notifications&email_token=AAUZW2WX7IBPJSNAZVSRPY3QUM5AZA5CNFSM4JOKMUC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEMP5ZA#issuecomment-555286244, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAUZW2SUDF7W76O5Q75AVCTQUM5AZANCNFSM4JOKMUCQ .

[Breakthrough]

Will revisit this for v0.6.1. Just some more general notes for the implementation under the new API.

This may need to only be conditionally enabled if ffprobe is available, and have a corresponding FAQ entry detailing the fix (which would be to install ffprobe). There should also be an option to disable the feature if it's auto-enabled, both via the CLI and the config file.

Your original function appeared as:

def get_rotation(file_path):
    # type: (str) -> str
    """ Get Rotation: Returns the number of degrees a video has been rotated.
    Uses subprocess to call external program ffprobe on the video.
    """
    cmd = '''ffprobe -loglevel error -select_streams v:0 -show_entries stream_tags=rotate \
             -of default=nw=1:nk=1 -i "%s"''' % file_path
    rotation = subprocess.getoutput(cmd)
    return rotation or None

This should probably go somewhere in the scenedetect.platform module, and then only be done in the actual save_images command/function. This avoids a potentially expensive rotation during the detection phase (video orientation shouldn't have an impact on detector performance).

Edit: Also after stumbling across the following: https://github.com/Zulko/moviepy/blob/2aff1cfc156292462bacc9859474d76546f9c03b/moviepy/video/io/ffmpeg_reader.py#L41= Investigate if using VideoStreamAv results in a different behavior.

[Breakthrough]

@markbaumgarten what version of OpenCV were you using when you ran into this? Can you try again with the latest version? It seems to be working all of the sudden without any changes on my end, at least when using OpenCV.

You can see the test I added for this here: https://github.com/Breakthrough/PySceneDetect/commit/d8b6e6cc206264e9ef508f46071d938ada834c7f#diff-4e8715c7a425ee52e74b7df4d34efd32e8c92f3e60bd51bc2e1ad5943b82032e=

This test fails when using PyAV or MoviePy instead of OpenCV for video decoding, so OpenCV does seem to be handling the rotation automatically now. Would prefer to not need to address this unless required for the other backends, so if this works now with OpenCV, I may close this and others can file bug reports for the other backends if they need this feature.

[Breakthrough]

Will mark this as complete for using the OpenCV backend (default) as this should work now with a new version of OpenCV.

If required on other backends please file a feature request. Thanks.