sparseml icon indicating copy to clipboard operation
sparseml copied to clipboard

Created base class for knowledge distillation

Open anmarques opened this issue 3 years ago • 0 comments

This PR creates a base class for knowledge distillation modifiers and changes the existing knowledge distillation modifier to inherit from the base class. The goal of this change is to enable other forms of knowledge distillation by simply implementing specialized instances of distillation losses.

anmarques avatar Jun 08 '22 02:06 anmarques