LYF

icon company NanChang University；TianJin University icon location NanChang；TianJin icon location Fish professional / loaf on the job

Results 6 issues of LYF

the TEACHER model used for the distillation of specific tasks

1 comment

May I ask if it is convenient to provide the TEACHER model used for the distillation of specific tasks, that is, the TEACHER model after fine-tuning of each task?

How is the softmax classifier initialized in the Bert-Base model?

How is the softmax classifier initialized in the Bert-Base model? Is zero initialized?

leopard/data/json/disaster/disaster_train_0_16.json is none

Do you have time to fill in this blanks?

looking for data

1 comment

Hello, I wonder if I can get your pre-processed brain data?

AttributeError: 'GraphModule' object has no attribute 'config'

when test quantization, it raises errors. May I ask if anyone has encountered this problem? pytorch==3.8.1 transformers==4.7.0

Request code for pretrain stage

Hi, would you like to publish the code of your pretrain stage? We are very much looking forward to further research based on this. Thanks!