UER-py icon indicating copy to clipboard operation
UER-py copied to clipboard
verified publisher icon dbiir

What's the hyperparameters to finetune Chinese bert-large on CLUE?

Open hitvoice opened this issue 5 years ago • 2 comments

Recently you updated "BERT pretrained on mixed large Chinese corpus (bert-large 24-layers) " on ReadMe. What hyperparameters (lr, batch size, max epochs) did you use when fine-tuning on CLUE?

hitvoice avatar Apr 26 '20 16:04 hitvoice

We basically use the following setting: lr=2e-5 batch_size=32, max_epochs=3 To reproduce the results on CLUE, we also need some other techniques which has been listed on CLUE leaderboard. We will open-source related code in the near future.

zhezhaoa avatar May 13 '20 03:05 zhezhaoa

Got it. Thanks for answering!

hitvoice avatar May 13 '20 07:05 hitvoice