Two different Data augmentations?

[priyarana]

Hi, According to the theory, in stage 1, two different types of augmentations should be applied to a batch of samples.

1. crop and resize
2. AutoAugment/randaugment/simaugment.

It seems the code doesn't have augmentation number (2). Are the mentioned results from only augmentation number (1) which has been applied twice to the same set of samples?

Thanks

[HobbitLong]

Hi, @priyarana,

Which theory are you referring to?

Yes, the mentioned results for CIFAR-10/100 only used (1).

[priyarana]

Thanks for the reply.

I was going through the following blog, which talks about two types of augmentations, https://wandb.ai/authors/scl/reports/Improving-Image-Classifiers-With-Supervised-Contrastive-Learning--VmlldzoxMzQwNzE

I am yet to read the whole paper, so not sure if using two different augmentations is the original theory! Thanks

[HobbitLong]

My understanding is two different augmentation means we apply the same augmentation strategy twice to a given image to get two different augmented copies of it. Though the augmentation strategy can be asymmetric for the two crops, people generally used the symmetric one.

[priyarana]

I'll see how results behave with these two different approaches. Thank you for the insights.

[priyarana]

I see the paper uses a batch size of 256, I am not sure if my GPU is going to afford that. Do you think the method works well with smaller batch sizes as well, such as 32 or 64? Thanks

[HobbitLong]

I think it still works, but the performance may drop (how much it will drop depends on the dataset).

[priyarana]

Sure, thanks. I am hoping to get better results than standard supervised learning which uses only cross-entropy.