bert_distill icon indicating copy to clipboard operation
bert_distill copied to clipboard

code疑问:BertModel的传参和distill中teacher的预测输入

Open Daishijun opened this issue 4 years ago • 0 comments

https://github.com/qiangsiwei/bert_distill/blob/ceed9c9455d70dde24990014945a382e290d61ff/ptbert.py#L103 这里的传参,input_mask是不是传错位置了,您这样相当于attention_mask是None, token_type_ids是input_mask。 https://github.com/qiangsiwei/bert_distill/blob/master/distill.py#L22 这里预测的时候为什么不加[CLS]和[SEP]了? 感谢大佬开源,希望大佬能解答下我的疑惑。

Daishijun avatar Oct 21 '20 03:10 Daishijun