bhheo

Hi, twmht Thank you for sharing the interesting aspect. I was surprised because one of my friends asks me the same question a few weeks ago. I think Adam optimizer...

@twmht Sorry for the late response. I thought the issue was closed because of the email alert. I recommend a cosine learning rate scheduler without cycles. https://github.com/rwightman/pytorch-image-models/blob/master/timm/scheduler/cosine_lr.py As I know,...

@zjykzj Thank you for your effort in implementation. It looks well-organized implementation with excellent results. It will help many researchers to compare their new distillation method with ours. I'm sorry...

Hi @TouchSkyWf Thank you for your interest in our research. I set the teacher network in training mode. But it doesn't mean the teacher network is trained in the distillation...