albumentations icon indicating copy to clipboard operation
albumentations copied to clipboard

[Documentation] Add examples how to apply transforms to videos

Open ternaus opened this issue 11 months ago • 4 comments

We can apply the same transform to videos using additional targets.

But no one knows about this methods => need example notebook

ternaus avatar Mar 05 '24 00:03 ternaus

Hello. I will have some free time starting the week of the 15th April and I would like to help with this. What type of open-source video dataset you would suggest for creating the tutorial notebook? Is there a similar notebook to get inspiration? Thanks. 😸

yassineAlouini avatar Apr 04 '24 08:04 yassineAlouini

Thanks!

Inspiration - it is almost this notebook: https://albumentations.ai/docs/examples/example_multi_target/

  1. Video is split into frames.
  2. For every frame we use example from the notebook to apply augmentation pipeline with the same parameters (could be very complicated). In the example about multi target we have:
import albumentations as A

transform = A.Compose(
    [HorizontalFlip(p=0.5), ...],
    additional_targets={
        'image1': 'image',
        'image2': 'image',
        ...
        'imageN': 'image',

        'bboxes1': 'bboxes',
        'bboxes1': 'bboxes',
        ...
        'bboxesM': 'bboxes',

        'keypoints1': 'keypoints',
        'keypoints2': 'keypoints',
        ...
        'keypointsK': 'keypoints',

        'mask1': 'mask',
        'mask2': 'mask',
        ...
        'maskL': 'mask'
    })
)

but make N = M = L = K = <the number of frames> and the definition of additional_targets dictionary could be done as a one liner.

For video - probably person / people dancing. This would allow to visualize on image, mask, bounding boxes + keypoints


Way that we create examples => we create notebooks to: https://github.com/albumentations-team/albumentations_examples and they are authomatically parsed to the documentation to the website.

If you create such a notebook and commit it to the albumentations_examples it would be great.

P.S. If you have questions - feel free to ask here, ask in Discord, or just ping me in DM

Thank you for helping!

ternaus avatar Apr 04 '24 19:04 ternaus

Thanks for the details @ternaus. Will make sure to check the Discord. 👌

yassineAlouini avatar Apr 06 '24 08:04 yassineAlouini

I have started working on the example. Hopefully will have a first version by next week. :ok_hand: (The branch in my fork: https://github.com/yassineAlouini/albumentations_examples/tree/feature/videos_transforms_example)

yassineAlouini avatar Apr 18 '24 16:04 yassineAlouini