Hang Zhang

icon company McGill University icon location Montreal icon location PhD candidate @ McGill Edge Intelligence Lab

Results 2 comments of Hang Zhang

请问General Distillation部分的teachermodel是什么，语料库如何获取？

The teacher model for general distillation is BERT-base-uncased and the corpus is the original one, Totonto Book Corpus. You can search pretrained BERT model on huggingface as reference.

TinyBERT's Google resource 404

Hi, did anyone find the open source pre-trained models? Thank you!