cls_KD icon indicating copy to clipboard operation
cls_KD copied to clipboard

Published 20 hours ago • verified publisher icon yzd-v

Questions on masked area

Open jwfanDL opened this issue 2 years ago • 0 comments

Hi Zhendong, In ViTKD, we only distill the knowledge from unmasked area, while full area in MGD. My questions are:

  1. Why ViTKD only distill the knowledge only from unmasked area
  2. What is the difference and relationship between unmasked and masked area in distillation.

jwfanDL avatar Oct 21 '22 05:10 jwfanDL