icefall icon indicating copy to clipboard operation
icefall copied to clipboard

Zipformer MVQ

Open marcoyang1998 opened this issue 2 years ago • 1 comments

This PR makes training with knowledge distillation an option in the Zipformer recipe. The knowledge distillation method is MVQ-KD.

The teacher targets can be downloaded via the following command:

./distillation_with_hubert.sh --stage 2 --stop_stage 2

To turn on knowledge distillation, you will need to set --enable-distillation True. It is applicable to both streaming and non-streaming Zipformers.

Detailed results will follow.

marcoyang1998 avatar Jul 28 '23 06:07 marcoyang1998

Some results:

100 hours:

model test-clean test-other
baseline, epoch-30-avg-9 5.97 15.73
+ mvq, epoch-30-avg-9 5.13 13.08

960 hours:

model test-clean test-other
baseline, epoch-30-avg-9 2.25 5.06
+ mvq, epoch-30-avg-9 2.18 4.86

marcoyang1998 avatar Aug 03 '23 02:08 marcoyang1998