Imagen segmentation

[Jcastanyo]

Hi everyone,

I wonder if It is possible to apply these methods in image segmentation. In that case, is there any tutorial available?

Thanks in advance. Julio.

[guarin]

Hi,

Self-supervised learning can definitely used for pretraining an image segmentation network. In this case it is important to consider how many objects per image your dataset contains. If you have a few large objects per image (lets say 1-3) then you could use for example one of our SimCLR or MoCo models.

If there are many small objects on the image it becomes more complicated as most self-supervised models assume that two random crops from the same image show the same object. This is no longer the case with small objects, as two crops from the same image could show very different objects. In this case you might want to change the cropping strategy such that the two crops always have some overlap. This approach is well explained in the following paper which also features many results on image segmentation: Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations

There have also been a flurry of new methods developed specifically for object detection and image segmentation on complex images. I put some examples down below:

• Efficient Visual Pretraining with Contrastive Detection
• Unsupervised Object-Level Representation Learning from Scene Images
• Aligning Pretraining for Detection via Object-Level Contrastive Learning

Sadly we did not yet have the time to implement those methods.

Could you provide some more details on what exactly you want to do and what type of data you have?

[miguel-arrf]

Hi!

@guarin, is there any Segmentation downtask example code available?

Thank you!

[philippmwirth]

Hi @miguel-arrf

Currently we unfortunately don't have any examples for segmentation downstream tasks.

[miguel-arrf]

Thank you @philippmwirth !