Pretrained-Language-Model icon indicating copy to clipboard operation
Pretrained-Language-Model copied to clipboard

Published 20 hours ago • verified publisher icon huawei-noah

请问General Distillation部分的teachermodel是什么，语料库如何获取？

Open YYangZiXin opened this issue 4 years ago • 1 comments

YYangZiXin avatar Oct 13 '20 03:10 YYangZiXin

The teacher model for general distillation is BERT-base-uncased and the corpus is the original one, Totonto Book Corpus. You can search pretrained BERT model on huggingface as reference.

janezzzz avatar Oct 23 '20 02:10 janezzzz