hexiaoyupku

question: How can I quantize BERT to FP16 ?

1

I have only P100 and V100 which dosen't support INT8. So what should I do to quantize BERT to FP16 ? Thanks in advance!

couldn't get the same accuracy of BERT when use no augmentation in UDA

3

Hi, thank you for your wonderful work! I tried UDA without any augmentation on my text classification task, I can only get 93% accuracy while BERT can get 96% accuracy...

Further pretrain in domain specific corpus

2

Hi, I tried to further pretrain XLNet in domain specific corpus like what is recommanded in BERT. But I got worse result. Has anyone tried further pretraining? Does it work?...