setfit
setfit copied to clipboard
Implement trainer for distillation
In our paper, we used knowledge distillation to compress a SetFit model into a smaller student. This logic is currently implemented as a script, but it would be valuable to refactor this out to a SetFitDistillationTrainer that people can use in e.g. a Jupyter notebook.
cc @orenpereg