overhaul-distillation

overhaul-distillation copied to clipboard

Does t_net also need to be trained?

1

d_net.train() d_net.module.s_net.train() d_net.module.t_net.train() Hi, I noticed that the Teacher model is also trained. In the general distillation process, shouldn't the teacher be in inference mode? Both in training mode will...

TouchSkyWf

A new implementation for overhual distillation

1

@ClovaAIAdmin Thank you for sharing this nice work Based on torchvision's ResNet/MobileNet and Distiller you realized, I make a new implementation [ZJCV/overhaul](https://github.com/ZJCV/overhaul) I have done some experiments, and can really...

zjykzj