RepDistiller icon indicating copy to clipboard operation
RepDistiller copied to clipboard

Hyperparameter Settings for KD on Imagenet

Open Calmepro777 opened this issue 2 years ago • 0 comments

To reproduce the baseline result on my machine (kd from rn34 to rn18), I would like to know the hyperparameter settings for knowledge distillation on Imagenet. Especially the weights for cross entropy loss and KLDiv loss, temperature, and batch_size. Thanks.

Calmepro777 avatar Aug 01 '22 05:08 Calmepro777