meets problem when train ssp head

[UcanSee]

Firstly I trained a teacher model and its accuracy is correct, then I train ssp head of teacher model, but I found loss of ssp head falling slowly. the initial loss is 3.37 at the start of training, and falls to 3.25 at the end of training. Did I make something wrong? dataset is ImageNet, and training config is consistent to that in student.py.

[larry10hhobh]

maybe the code of train ssp head is wrong? you can read my issue.

[xuguodong03]

The training hyper-parameters (e.g. batchsize, epoch, LR) of CIFAR and ImageNet are different. For ImageNet, we use the hyper-parameters in pytorch/example.

Besides the hyper-parameters, the reason that ssp loss does not fall may be that the backbone of teacher is fixed. The trainable module contains only a 2-layer FC. As stated in the paper, the self-supervision may be not accurate, but it still transfer some structured information. So maybe you could try continuing the experiments and see the results.