albumentations CutMix and Mosaic Augmentation

CutMix and Mosaic Augmentation

Open awsaf49 opened this issue 3 years ago • 15 comments

📚 Documentation

CutMix and Mosaic Augmentations are pretty good augmentation when it comes to achieve better score. It would be great if these two are included in albumentations.

Aug 10 '20 23:08 awsaf49

Could you please provide links to the description of Mosaic Augmentation?

Aug 10 '20 23:08 ternaus

Could you please provide links to the description of Mosaic Augmentation?

I think it was even mentioned in YOLOv4 paper. Here are some references: https://www.kaggle.com/nvnnghia/awesome-augmentation https://github.com/klauspa/Yolov4-tensorflow

Aug 11 '20 00:08 awsaf49

Is someone working on this issue ? Can I pick this up and create a PR for this request ?

Jul 29 '21 05:07 chamorajg

Yes, you can try to implement this.

Jul 29 '21 07:07 Dipet

I guess what makes it difficult is that you would need to implement an augmentation that takes multiple images as input. Is that even possible in Albumentations?

Aug 26 '21 06:08 senarvi

The BasicTransform (and all other transforms) take only 1 image a input and don't manage iterables themselves. So I don't think it's possible without changing how albumentations or putting them as a different kind of object.

We would need to create an "BasicIterableTransform" than takes an iterable as input. But it wouldn't work with tools like torch.utils.data.Dataset without modification, need a new compose class, etc.

I agree it's a shame they're not included. It's super useful in detection problems, even though it requires quite a few modifications in albumentations and changes how the transforms would be used.

Any maintainer can tell us if it's the kind of features (namely Transforms taking iterables) that would be accepted/merged or it's just too different to how the library works now?

Nov 22 '21 15:11 mulcyber

I've been working on a custom augment that performs mosaic by loading in 3 new images with whatever image is currently in the pipeline, but I got stuck at trying to return the new set of bounding boxes in the final moasic-ed image. Has anyone made progress on this?

Mar 14 '22 18:03 dav-ell

I was able to create a custom version that fit my needs, but I couldn't get it to work with the albumentations pipeline. I ended up doing this (where self.mosaic :

# in my datamodule

self.mosaic = Mosaic(img_files, bbox_files, p=0.9) # requires all image and label files to create random mosaics

def transform(image, bboxes, class_labels, **params):
    """A wrapper for the albumentations pipeline"""
    image, bboxes, class_labels = self.mosaic(image, bboxes, class_labels)
    return self.transform(image=image, bboxes=bboxes, class_labels=class_labels, **params)

# create dataset, pass transform

That way my dataset could still call transform() with the usual albumentations API. However, it'd be nice to not have to use workarounds like this.

Mar 15 '22 15:03 dav-ell

I made a PR to add Mosaic augmentation with as few changes to the existing implementation as possible #1147. Although some remaining works are to be done, please check it out if you are interested in this feature. I welcome any comments and feedback.