setfit icon indicating copy to clipboard operation
setfit copied to clipboard

Implement trainer for distillation

Open lewtun opened this issue 3 years ago • 0 comments

In our paper, we used knowledge distillation to compress a SetFit model into a smaller student. This logic is currently implemented as a script, but it would be valuable to refactor this out to a SetFitDistillationTrainer that people can use in e.g. a Jupyter notebook.

cc @orenpereg

lewtun avatar Oct 07 '22 09:10 lewtun