what is ShiftConvDownsample in ResNext and shakeshake26

[liangzimei]

hi , firstly thanks for your great work for ssl. But when i refer many resnext nets of pytorch, there are no ShiftConvDownsample layer? what is the function of it? And mean teacher didn't use this layer in the experiment of cifar10 and imagenet, right? And the two fc layers after avepooling correspond to student and teacher? thanks in advance...

[tarvaina]

Hi,

I reimplemented the architecture from the Shake-shake regularization paper (which was the state of the art on CIFAR-10 dataset at the time of the writing of Mean Teacher), and they had this special downsampling layer, which apparently improves the results somewhat. It's not fundamental to Mean Teacher or even the shake-shake regularization as I've understood it.

The ResNet CIFAR-10 experiments do use the layer, the ImageNet experiments do not.

Antti

[liangzimei]

@tarvaina thank you for the detailed explanation, i got it. :)