Reproducing the results on transfer tasks

[zybeich]

Thanks a lot for your brilliant work. I'm trying to use your ImageNet pre-trained model dissl_resnet50_dNone_e400_m6 to train a linear probing and evaluate it on CIFAR-10. I followed the instructions in F.2. Featurized the data, and min-max scaled it into [0, 1]. Then use both logistic regression and SVC with parameter C to train it. The best result of both on the test set is about 83%. I attached the accuracies I got below.

reg	LR_train	LR_test	SVC_train	SVC_test
-4.00	64.32	63.63	74.23	73.45
-3.33	70.58	70.11	80.16	79.14
-2.67	77.30	76.03	84.56	82.42
-2.00	82.62	80.84	87.75	83.66
-1.33	86.76	83.23	89.80	83.73
-0.67	90.16	83.96	91.08	82.81
0.00	92.81	83.19	91.63	81.75
0.67	94.62	81.62	91.85	80.76
1.33	95.70	79.57	89.64	78.35
2.00	96.36	78.04	86.53	76.69

[YannDubs]

That's strange... The model that I used to get table 5 (ie 93.6 CIFAR10) is dissl_resnet50_d8192_e400_m6, to check that you can reproduce the results?

[zybeich]

I just tried dissl_resnet50_d8192_e400_m6. The accuracy increased by 2 percent. I evaluated this model by change the dataset to CIFAR10 in the code your uploaded torchhub. I also evaluate it by following the instruction (add a min-max scaler). Both get the similar result (85%)